PrivateCoffee/structables
4
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Strange 500 errors #9

New issue
Closed
opened 2024-10-01 17:13:28 +00:00 by vlnst · 19 comments
vlnst commented 2024-10-01 17:13:28 +00:00
Contributor
Copy link

For some reason on some URLs I get 500 errors.

Example:
https://structables.bloat.cat/MechaMaven-the-Educational-Robot-Explorer/
https://structables.bloat.cat/contest/
https://structables.bloat.cat/teachers/

It works on official instance though.

Here are some log messages:

ERROR:structables.main:Exception on /teachers/ [GET]
Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/structables/routes/category.py", line 61, in route_teachers
    return category_page(app, "Teachers", True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/structables/utils/helpers.py", line 302, in category_page
    for channel in app.global_ibles["/projects"]:
                   ^^^^^^^^^^^^^^^^
AttributeError: 'Flask' object has no attribute 'global_ibles'

[pid: 16|app: 0|req: 2/9] 172.17.0.1 () {28 vars in 477 bytes} [Tue Oct  1 17:10:58 2024] GET /Elastic-Band-Shooter/ => generated 9616 bytes in 731 msecs (HTTP/1.0 200) 2 headers in 81 bytes (1 switches on core 3)
DEBUG:root:Generating proxy URL for https://content.instructables.com/FA0/FHZ5/LWYZDP97/FA0FHZ5LWYZDP97.jpg
Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/structables/routes/main.py", line 167, in route_article
    "src": proxy(file["downloadUrl"], file["name"]),
                 ~~~~^^^^^^^^^^^^^^^
KeyError: 'downloadUrl'
[pid: 12|app: 0|req: 5/10] 172.17.0.1 () {42 vars in 780 bytes} [Tue Oct  1 17:10:59 2024] GET /MechaMaven-the-Educational-Robot-Explorer/ => generated 2449 bytes in 55 msecs (HTTP/1.0 500) 2 headers in 100 bytes (1 switches on core 0)
[pid: 16|app: 0|req: 3/11] 172.17.0.1 () {42 vars in 692 bytes} [Tue Oct  1 17:10:59 2024] GET /static/css/style.css => generated 5327 bytes in 11 msecs via sendfile() (HTTP/1.0 200) 7 headers in 275 bytes (0 switches on core 2)
[pid: 12|app: 0|req: 6/12] 172.17.0.1 () {42 vars in 718 bytes} [Tue Oct  1 17:10:59 2024] GET /static/dist/css/bootstrap.min.css => generated 232948 bytes in 12 msecs via sendfile() (HTTP/1.0 200) 7 headers in 286 bytes (1 switches on core 2)
[pid: 11|app: 0|req: 4/13] 172.17.0.1 () {42 vars in 744 bytes} [Tue Oct  1 17:10:59 2024] GET /static/img/logo.png => generated 24209 bytes in 10 msecs via sendfile() (HTTP/1.0 200) 7 headers in 262 bytes (0 switches on core 0)
[pid: 11|app: 0|req: 4/14] 172.17.0.1 () {28 vars in 715 bytes} [Tue Oct  1 17:10:59 2024] GET /proxy/?filename=Hook.SLDPRT&url=https%3A%2F%2Fcontent.instructables.com%2FFLL%2FJLRJ%2FLGUUMPCZ%2FFLLJLRJLGUUMPCZ.sldprt => generated 188047 bytes in 293 msecs (HTTP/1.0 200) 2 headers in 116 bytes (1 switches on core 3)
[pid: 16|app: 0|req: 4/15] 172.17.0.1 () {28 vars in 463 bytes} [Tue Oct  1 17:11:02 2024] GET /Elderberry-Jam => generated 263 bytes in 1 msecs (HTTP/1.0 308) 3 headers in 146 bytes (1 switches on core 3)
ERROR:structables.main:Exception on /contest/ [GET]
Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/structables/routes/contest.py", line 123, in route_contests
    contest_count = str(soup.select("p.contest-count")[0])
                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

I'm using Docker image; couldn't figure what is wrong 😞

For some reason on some URLs I get 500 errors. Example: https://structables.bloat.cat/MechaMaven-the-Educational-Robot-Explorer/ https://structables.bloat.cat/contest/ https://structables.bloat.cat/teachers/ It works on official instance though. Here are some log messages: ``` ERROR:structables.main:Exception on /teachers/ [GET] Traceback (most recent call last): File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app response = self.full_dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request rv = self.handle_user_exception(e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request rv = self.dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/structables/routes/category.py", line 61, in route_teachers return category_page(app, "Teachers", True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/structables/utils/helpers.py", line 302, in category_page for channel in app.global_ibles["/projects"]: ^^^^^^^^^^^^^^^^ AttributeError: 'Flask' object has no attribute 'global_ibles' ``` ``` [pid: 16|app: 0|req: 2/9] 172.17.0.1 () {28 vars in 477 bytes} [Tue Oct 1 17:10:58 2024] GET /Elastic-Band-Shooter/ => generated 9616 bytes in 731 msecs (HTTP/1.0 200) 2 headers in 81 bytes (1 switches on core 3) DEBUG:root:Generating proxy URL for https://content.instructables.com/FA0/FHZ5/LWYZDP97/FA0FHZ5LWYZDP97.jpg Traceback (most recent call last): File "/opt/venv/lib/python3.12/site-packages/structables/routes/main.py", line 167, in route_article "src": proxy(file["downloadUrl"], file["name"]), ~~~~^^^^^^^^^^^^^^^ KeyError: 'downloadUrl' [pid: 12|app: 0|req: 5/10] 172.17.0.1 () {42 vars in 780 bytes} [Tue Oct 1 17:10:59 2024] GET /MechaMaven-the-Educational-Robot-Explorer/ => generated 2449 bytes in 55 msecs (HTTP/1.0 500) 2 headers in 100 bytes (1 switches on core 0) [pid: 16|app: 0|req: 3/11] 172.17.0.1 () {42 vars in 692 bytes} [Tue Oct 1 17:10:59 2024] GET /static/css/style.css => generated 5327 bytes in 11 msecs via sendfile() (HTTP/1.0 200) 7 headers in 275 bytes (0 switches on core 2) [pid: 12|app: 0|req: 6/12] 172.17.0.1 () {42 vars in 718 bytes} [Tue Oct 1 17:10:59 2024] GET /static/dist/css/bootstrap.min.css => generated 232948 bytes in 12 msecs via sendfile() (HTTP/1.0 200) 7 headers in 286 bytes (1 switches on core 2) [pid: 11|app: 0|req: 4/13] 172.17.0.1 () {42 vars in 744 bytes} [Tue Oct 1 17:10:59 2024] GET /static/img/logo.png => generated 24209 bytes in 10 msecs via sendfile() (HTTP/1.0 200) 7 headers in 262 bytes (0 switches on core 0) [pid: 11|app: 0|req: 4/14] 172.17.0.1 () {28 vars in 715 bytes} [Tue Oct 1 17:10:59 2024] GET /proxy/?filename=Hook.SLDPRT&url=https%3A%2F%2Fcontent.instructables.com%2FFLL%2FJLRJ%2FLGUUMPCZ%2FFLLJLRJLGUUMPCZ.sldprt => generated 188047 bytes in 293 msecs (HTTP/1.0 200) 2 headers in 116 bytes (1 switches on core 3) [pid: 16|app: 0|req: 4/15] 172.17.0.1 () {28 vars in 463 bytes} [Tue Oct 1 17:11:02 2024] GET /Elderberry-Jam => generated 263 bytes in 1 msecs (HTTP/1.0 308) 3 headers in 146 bytes (1 switches on core 3) ERROR:structables.main:Exception on /contest/ [GET] Traceback (most recent call last): File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app response = self.full_dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request rv = self.handle_user_exception(e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request rv = self.dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/structables/routes/contest.py", line 123, in route_contests contest_count = str(soup.select("p.contest-count")[0]) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^ IndexError: list index out of range ``` I'm using Docker image; couldn't figure what is wrong 😞
kumi commented 2024-10-02 10:24:14 +00:00
Owner
Copy link

I believe you are seeing two different issues there. I think I know what is causing the first – your instance probably never initialized its cache. I have released a new version, v0.3.12, which should address this. Simply loading /cron/ made the /teachers/ category load fine for me on your instance.

The first issue in your second code block looks like you are not on the latest release:

File "/opt/venv/lib/python3.12/site-packages/structables/routes/main.py", line 167, in route_article
"src": proxy(file["downloadUrl"], file["name"]),
~~~~^^^^^^^^^^^^^^^
KeyError: 'downloadUrl'

This has already been fixed in e845caff0e, which was included in the previous v0.3.11 release (in fact, that was the only change in that release). Please try updating to v0.3.12 (a.k.a. latest in Docker Hub), or the latest commit if you are building the container yourself.

The third issue is due to a change in the structure of the Instructables website, which will have to be reflected in the Structables code. I will move that part to a separate issue.

I believe you are seeing two different issues there. I think I know what is causing the first – your instance probably never initialized its cache. I have released a new version, `v0.3.12`, which should address this. Simply loading `/cron/` made the `/teachers/` category load fine for me on your instance. The first issue in your second code block looks like you are not on the latest release: > File "/opt/venv/lib/python3.12/site-packages/structables/routes/main.py", line 167, in route_article > "src": proxy(file["downloadUrl"], file["name"]), > ~~~~^^^^^^^^^^^^^^^ > KeyError: 'downloadUrl' This has already been fixed in e845caff0e3b560eb0bdff0d849d22a558249695, which was included in the previous `v0.3.11` release (in fact, that was the only change in that release). Please try updating to `v0.3.12` (a.k.a. `latest` in Docker Hub), or the latest commit if you are building the container yourself. The third issue is due to a change in the structure of the Instructables website, which will have to be reflected in the Structables code. I will move that part to a separate issue.
kumi closed this issue 2024-10-02 10:24:14 +00:00
vlnst commented 2024-10-02 10:44:05 +00:00
Author
Contributor
Copy link

Pulled latest image from DockerHub, still have this issue.

Removed info about /contest/

This is on /projects/

ERROR:structables.main:Exception on /projects/ [GET]
Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/structables/routes/category.py", line 29, in route_projects
    return project_list(app, "")
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/structables/utils/helpers.py", line 214, in project_list
    ibles = app.global_ibles["/projects"]
            ^^^^^^^^^^^^^^^^
AttributeError: 'Flask' object has no attribute 'global_ibles'

/teachers/ works fine.

Pulled latest image from DockerHub, still have this issue. ~Removed info about /contest/~ This is on /projects/ ``` ERROR:structables.main:Exception on /projects/ [GET] Traceback (most recent call last): File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app response = self.full_dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request rv = self.handle_user_exception(e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request rv = self.dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/structables/routes/category.py", line 29, in route_projects return project_list(app, "") ^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/structables/utils/helpers.py", line 214, in project_list ibles = app.global_ibles["/projects"] ^^^^^^^^^^^^^^^^ AttributeError: 'Flask' object has no attribute 'global_ibles' ``` /teachers/ works fine.
vlnst reopened this issue 2024-10-02 10:44:06 +00:00
vlnst commented 2024-10-02 10:46:25 +00:00
Author
Contributor
Copy link

I'm getting random 500 errors. Sometimes everything loads fine if I try reloading again and again

I'm getting random 500 errors. Sometimes everything loads fine if I try reloading again and again
kumi commented 2024-10-02 10:56:02 +00:00
Owner
Copy link

Hmm, that still sounds like it's related to cache initialization. But I can't reproduce the issue on your instance right now - no matter how often I try to load /projects/, it loads perfectly fine each time.

Hmm, that still sounds like it's related to cache initialization. But I can't reproduce the issue on your instance right now - no matter how often I try to load `/projects/`, it loads perfectly fine each time.
vlnst commented 2024-10-02 11:01:38 +00:00
Author
Contributor
Copy link

Tried on another computer with Windows 10 in Chrome, it is less frequent than on Linux in Firefox but I still managed to get 500 error

Tried on another computer with Windows 10 in Chrome, it is less frequent than on Linux in Firefox but I still managed to get 500 error
vlnst commented 2024-10-02 11:18:32 +00:00
Author
Contributor
Copy link

I think you need to spam so it gets 500... Probably not the issue then.

/privacypolicy/ is not loading too

I think you need to spam so it gets 500... Probably not the issue then. /privacypolicy/ is not loading too
kumi referenced this issue from a commit 2024-10-04 05:08:49 +00:00
chore: bump version to 0.3.13
kumi commented 2024-10-04 05:13:07 +00:00
Owner
Copy link

/privacypolicy/ is not loading too

Do you have logs for that?

> /privacypolicy/ is not loading too Do you have logs for that?
vlnst commented 2024-10-04 11:21:14 +00:00
Author
Contributor
Copy link

Do you have logs for that?

Sure, here it is:

[pid: 19|app: 0|req: 1/3] 172.17.0.1 () {46 vars in 794 bytes} [Fri Oct  4 11:19:49 2024] GET /privacypolicy => generated 261 bytes in 5 msecs (HTTP/1.0 308) 3 headers in 145 bytes (1 switches on core 0)
ERROR:structables.main:Exception on /privacypolicy/ [GET]
Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/structables/routes/main.py", line 361, in privacypolicy
    with pathlib.Path(path).open() as f:
         ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/pathlib.py", line 1162, in __init__
    super().__init__(*args)
  File "/usr/lib/python3.12/pathlib.py", line 373, in __init__
    raise TypeError(
TypeError: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType'
> Do you have logs for that? Sure, here it is: ```python [pid: 19|app: 0|req: 1/3] 172.17.0.1 () {46 vars in 794 bytes} [Fri Oct 4 11:19:49 2024] GET /privacypolicy => generated 261 bytes in 5 msecs (HTTP/1.0 308) 3 headers in 145 bytes (1 switches on core 0) ERROR:structables.main:Exception on /privacypolicy/ [GET] Traceback (most recent call last): File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app response = self.full_dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request rv = self.handle_user_exception(e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request rv = self.dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/structables/routes/main.py", line 361, in privacypolicy with pathlib.Path(path).open() as f: ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/pathlib.py", line 1162, in __init__ super().__init__(*args) File "/usr/lib/python3.12/pathlib.py", line 373, in __init__ raise TypeError( TypeError: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType' ```
vlnst commented 2024-10-04 11:33:58 +00:00
Author
Contributor
Copy link

/projects/

ERROR:structables.main:Exception on /projects/ [GET]
Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/structables/routes/category.py", line 29, in route_projects
    return project_list(app, "")
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/structables/utils/helpers.py", line 214, in project_list
    ibles = app.global_ibles["/projects"]
            ^^^^^^^^^^^^^^^^
AttributeError: 'Flask' object has no attribute 'global_ibles'

/teachers/

ERROR:structables.main:Exception on /teachers/ [GET]
Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/structables/routes/category.py", line 61, in route_teachers
    return category_page(app, "Teachers", True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/structables/utils/helpers.py", line 302, in category_page
    for channel in app.global_ibles["/projects"]:
                   ^^^^^^^^^^^^^^^^
AttributeError: 'Flask' object has no attribute 'global_ibles'

/contest/

ERROR:structables.main:Exception on /contest/ [GET]
Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/structables/routes/contest.py", line 123, in route_contests
    contest_count = str(soup.select("p.contest-count")[0])
                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

If I keep spamming reload button, eventually I'll get the page, but once reloaded, I'll get 500 again (/teachers/ and /projects/ can load this way, /privacypolicy/ and /contests/ aren't loaded even if I spam)

/projects/ ```python ERROR:structables.main:Exception on /projects/ [GET] Traceback (most recent call last): File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app response = self.full_dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request rv = self.handle_user_exception(e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request rv = self.dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/structables/routes/category.py", line 29, in route_projects return project_list(app, "") ^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/structables/utils/helpers.py", line 214, in project_list ibles = app.global_ibles["/projects"] ^^^^^^^^^^^^^^^^ AttributeError: 'Flask' object has no attribute 'global_ibles' ``` /teachers/ ```python ERROR:structables.main:Exception on /teachers/ [GET] Traceback (most recent call last): File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app response = self.full_dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request rv = self.handle_user_exception(e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request rv = self.dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/structables/routes/category.py", line 61, in route_teachers return category_page(app, "Teachers", True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/structables/utils/helpers.py", line 302, in category_page for channel in app.global_ibles["/projects"]: ^^^^^^^^^^^^^^^^ AttributeError: 'Flask' object has no attribute 'global_ibles' ``` /contest/ ```python ERROR:structables.main:Exception on /contest/ [GET] Traceback (most recent call last): File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app response = self.full_dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request rv = self.handle_user_exception(e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request rv = self.dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/structables/routes/contest.py", line 123, in route_contests contest_count = str(soup.select("p.contest-count")[0]) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^ IndexError: list index out of range ``` If I keep spamming reload button, eventually I'll get the page, but once reloaded, I'll get 500 again (/teachers/ and /projects/ can load this way, /privacypolicy/ and /contests/ aren't loaded even if I spam)
kumi commented 2024-10-04 11:39:54 +00:00
Owner
Copy link

Do you have logs for that?

Sure, here it is:

[pid: 19|app: 0|req: 1/3] 172.17.0.1 () {46 vars in 794 bytes} [Fri Oct  4 11:19:49 2024] GET /privacypolicy => generated 261 bytes in 5 msecs (HTTP/1.0 308) 3 headers in 145 bytes (1 switches on core 0)
ERROR:structables.main:Exception on /privacypolicy/ [GET]
Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/structables/routes/main.py", line 361, in privacypolicy
    with pathlib.Path(path).open() as f:
         ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/pathlib.py", line 1162, in __init__
    super().__init__(*args)
  File "/usr/lib/python3.12/pathlib.py", line 373, in __init__
    raise TypeError(
TypeError: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType'

Alright, that one should be trivial to fix, I just have to catch that exception. I'll do that on the train later. Although you should probably provide a privacy policy.

> > Do you have logs for that? > > Sure, here it is: > ```python > [pid: 19|app: 0|req: 1/3] 172.17.0.1 () {46 vars in 794 bytes} [Fri Oct 4 11:19:49 2024] GET /privacypolicy => generated 261 bytes in 5 msecs (HTTP/1.0 308) 3 headers in 145 bytes (1 switches on core 0) > ERROR:structables.main:Exception on /privacypolicy/ [GET] > Traceback (most recent call last): > File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app > response = self.full_dispatch_request() > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request > rv = self.handle_user_exception(e) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request > rv = self.dispatch_request() > ^^^^^^^^^^^^^^^^^^^^^^^ > File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request > return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/opt/venv/lib/python3.12/site-packages/structables/routes/main.py", line 361, in privacypolicy > with pathlib.Path(path).open() as f: > ^^^^^^^^^^^^^^^^^^ > File "/usr/lib/python3.12/pathlib.py", line 1162, in __init__ > super().__init__(*args) > File "/usr/lib/python3.12/pathlib.py", line 373, in __init__ > raise TypeError( > TypeError: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType' > ``` Alright, that one should be trivial to fix, I just have to catch that exception. I'll do that on the train later. Although you should probably provide a privacy policy.
kumi commented 2024-10-04 11:40:35 +00:00
Owner
Copy link

If I keep spamming reload button, eventually I'll get the page, but once reloaded, I'll get 500 again (/teachers/ and /projects/ can load this way, /privacypolicy/ and /contests/ aren't loaded even if I spam)

Question: Do you load ˋ/cron/ˋ, like, ever?

> If I keep spamming reload button, eventually I'll get the page, but once reloaded, I'll get 500 again (/teachers/ and /projects/ can load this way, /privacypolicy/ and /contests/ aren't loaded even if I spam) Question: Do you load ˋ/cron/ˋ, like, ever?
kumi commented 2024-10-04 11:44:32 +00:00
Owner
Copy link

And do you see "Updating data..." at any point in your logs?

And do you see "Updating data..." at any point in your logs?
vlnst commented 2024-10-04 12:16:18 +00:00
Author
Contributor
Copy link

If I keep spamming reload button, eventually I'll get the page, but once reloaded, I'll get 500 again (/teachers/ and /projects/ can load this way, /privacypolicy/ and /contests/ aren't loaded even if I spam)

Question: Do you load ˋ/cron/ˋ, like, ever?

Yes, if I don't load then I think I'll never get /teachers/ and /projects/ to load, so everytime I restart a container I go to /cron/ page

> > If I keep spamming reload button, eventually I'll get the page, but once reloaded, I'll get 500 again (/teachers/ and /projects/ can load this way, /privacypolicy/ and /contests/ aren't loaded even if I spam) > > Question: Do you load ˋ/cron/ˋ, like, ever? Yes, if I don't load then I think I'll never get /teachers/ and /projects/ to load, so everytime I restart a container I go to /cron/ page
kumi commented 2024-10-04 12:36:18 +00:00
Owner
Copy link

Yes, if I don't load then I think I'll never get /teachers/ and /projects/ to load, so everytime I restart a container I go to /cron/ page

Then there's your issue. ˋ/cron/ˋ has to be run regularly, ideally, but at least once on each worker thread. The individual uwsgi workers don't have a shared state/dataset.

Do you see "Updating data..." in your logs before you manually load ˋ/cron/ˋ? Since the last release, the cron function should be executed automatically on start of a new worker.

> Yes, if I don't load then I think I'll never get /teachers/ and /projects/ to load, so everytime I restart a container I go to /cron/ page Then there's your issue. ˋ/cron/ˋ has to be run regularly, ideally, but *at least once on each worker thread*. The individual uwsgi workers don't have a shared state/dataset. Do you see "Updating data..." in your logs *before* you manually load ˋ/cron/ˋ? Since the last release, the cron function should be executed automatically on start of a new worker.
kumi commented 2024-10-04 13:27:54 +00:00
Owner
Copy link

I have created a new release with a fix for the privacy policy issue and moved the data loading to an earlier point in the initialization, that should ensure that every new worker thread comes with data to display. Could you please test?

I have created a new release with a fix for the privacy policy issue and moved the data loading to an earlier point in the initialization, that should ensure that every new worker thread comes with data to display. Could you please test?
vlnst commented 2024-10-04 13:46:32 +00:00
Author
Contributor
Copy link

And do you see "Updating data..." at any point in your logs?

Yes, I do.

Do you see "Updating data..." in your logs before you manually load ˋ/cron/ˋ? Since the last release, the cron function should be executed automatically on start of a new worker.

Yes, It seems it works automatically, but not as frequent...

Pulled the latest image, everything is fine exept I'm still getting an error on /priacypolicy/

ERROR:structables.main:Exception on /privacypolicy/ [GET]
Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/structables/routes/main.py", line 361, in privacypolicy
    with pathlib.Path(path).open() as f:
         ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/pathlib.py", line 1162, in __init__
    super().__init__(*args)
  File "/usr/lib/python3.12/pathlib.py", line 373, in __init__
    raise TypeError(
TypeError: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType'

Although you should probably provide a privacy policy.

How do I do it?

> And do you see "Updating data..." at any point in your logs? Yes, I do. > Do you see "Updating data..." in your logs before you manually load ˋ/cron/ˋ? Since the last release, the cron function should be executed automatically on start of a new worker. Yes, It seems it works automatically, but not as frequent... **Pulled the latest image, everything is fine** exept I'm still getting an error on /priacypolicy/ ```python ERROR:structables.main:Exception on /privacypolicy/ [GET] Traceback (most recent call last): File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app response = self.full_dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request rv = self.handle_user_exception(e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request rv = self.dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/structables/routes/main.py", line 361, in privacypolicy with pathlib.Path(path).open() as f: ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/pathlib.py", line 1162, in __init__ super().__init__(*args) File "/usr/lib/python3.12/pathlib.py", line 373, in __init__ raise TypeError( TypeError: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType' ``` > Although you should probably provide a privacy policy. How do I do it?
kumi commented 2024-10-04 14:10:51 +00:00
Owner
Copy link

How do I do it?

Ah, you use Docker, right? No clue, to be honest, never tested that. I assume you'd have to bind (is that the word?) a privacy.txt or privacy.md into the working directory in the container.

File "/opt/venv/lib/python3.12/site-packages/structables/routes/main.py", line 361, in privacypolicy
with pathlib.Path(path).open() as f:

Erm. You're not running the latest code. That's line 362 now, not 361.

> How do I do it? Ah, you use Docker, right? No clue, to be honest, never tested that. I assume you'd have to bind (is that the word?) a privacy.txt or privacy.md into the working directory in the container. > File "/opt/venv/lib/python3.12/site-packages/structables/routes/main.py", line 361, in privacypolicy > with pathlib.Path(path).open() as f: Erm. You're not running the latest code. That's line 362 now, not 361.
vlnst commented 2024-10-04 14:35:30 +00:00
Author
Contributor
Copy link

Ah, you use Docker, right? No clue, to be honest, never tested that. I assume you'd have to bind (is that the word?) a privacy.txt or privacy.md into the working directory in the container.

I tried binding privacy.txt to /opt/venv/privacy.txt but I don't this this is the right way since /privacypolicy/ doesn't show it I guess I need to set STRUCTABLES_PRIVACY_FILE env variable for it to work. Update: it works! This info should be in README.md I think, not in the code.

Erm. You're not running the latest code. That's line 362 now, not 361.

Sorry, my bad. Now everything is fine. Closing the issue

> Ah, you use Docker, right? No clue, to be honest, never tested that. I assume you'd have to bind (is that the word?) a privacy.txt or privacy.md into the working directory in the container. ~I tried binding privacy.txt to /opt/venv/privacy.txt but I don't this this is the right way since /privacypolicy/ doesn't show it~ I guess I need to set STRUCTABLES_PRIVACY_FILE env variable for it to work. Update: it works! This info should be in README.md I think, not in the code. > Erm. You're not running the latest code. That's line 362 now, not 361. Sorry, my bad. Now everything is fine. Closing the issue
vlnst closed this issue 2024-10-04 14:35:31 +00:00
kumi commented 2024-10-04 14:48:23 +00:00
Owner
Copy link

That info wasn't even in my brain anymore. 😂 I'll add it to the README.md.

Anyway, I think the working directory in a Docker container is / by default, so moving it there should work as well.

That info wasn't even in my brain anymore. 😂 I'll add it to the README.md. Anyway, I think the working directory in a Docker container is `/` by default, so moving it there should work as well.
👍 1
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
v0.3.15
v0.3.14
v0.3.13
v0.3.12
v0.3.11
v0.3.10
v0.3.9
v0.3.8
v0.3.7
v0.3.6
v0.3.5
v0.3.4
v0.2.1
v0.2.0
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: PrivateCoffee/structables#9
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?