feat: Upgrade project setup and dependencies management

This commit represents a significant overhaul of the development and
setup processes for the Quackscape project. Key changes include the
transition from a requirements.txt-based dependency management system to
Poetry.
These updates aim to streamline the setup process

- Transitioned to Poetry for improved dependency resolution and package
management.
- Consolidated Django management commands under the quackscape-manage
script to simplify command execution and project management.
- Updated README.md to reflect the new setup procedures, emphasizing
ease of setup and the flexibility to support multiple database backends.

The shift to Poetry and the introduction of a custom management command
tool reflects an effort to modernize the project's infrastructure,
making it more accessible to contributors and easier to maintain.
This commit is contained in:
Kumi 2024-03-15 09:36:18 +01:00
parent 743f97a6ab
commit 8c3ea535e5
Signed by: kumi
GPG key ID: ECBCC9082395383F
4 changed files with 86 additions and 32 deletions

View file

@ -13,14 +13,25 @@ Quackscape is a content management system for panoramic/VR photos and videos. It
## Development Setup ## Development Setup
1. Clone the repository 1. Clone the repository and enter it
```bash
git clone https://git.private.coffee/PrivateCoffee/quackscape.git
cd quackscape
```
2. Create a virtual environment and install the requirements 2. Create a virtual environment and install the requirements
```bash ```bash
python3 -m venv venv python3 -m venv venv
source venv/bin/activate source venv/bin/activate
pip install -r requirements.txt pip install -e .
# Or, to install with MySQL support:
pip install -e .[mysql]
# Or for Postgres support:
pip install -e .[postgres]
``` ```
3. Install the frontend dependencies 3. Install the frontend dependencies
@ -34,27 +45,27 @@ npm install
5. Run the migrations 5. Run the migrations
```bash ```bash
python manage.py migrate quackscape-manage migrate
``` ```
6. Compile the frontend assets 6. Compile the frontend assets
```bash ```bash
npm run build:dev quackscape-manage build
``` ```
If you are working on the frontend, you can instead use the `npm run watch:dev` command to automatically recompile the frontend assets when they change. If you are working on the frontend, you can instead use the `npm run watch:dev` command to automatically recompile the frontend assets when they change. Note that if you do this, you will have to keep this running while you are working, so it will probably occupy one of your terminals.
7. Run the development server 7. Run the development server
```bash ```bash
python manage.py rundev quackscape-manage rundev
``` ```
8. Create a superuser 8. Create a superuser
```bash ```bash
python manage.py createsuperuser quackscape-manage createsuperuser
``` ```
9. Visit `http://localhost:8000` in your web browser 9. Visit `http://localhost:8000` in your web browser
@ -63,15 +74,28 @@ python manage.py createsuperuser
As this is still quite a ways from a stable project, we do not include production setup steps here. Very fundamentally, the production setup would be similar to the development setup. However, there are a few differences: As this is still quite a ways from a stable project, we do not include production setup steps here. Very fundamentally, the production setup would be similar to the development setup. However, there are a few differences:
- You can omit the `-e` switch when running `pip install`.
- Instead of `npm run build:dev`, you should run `npm run build`. - Instead of `npm run build:dev`, you should run `npm run build`.
- You should make sure to configure a database such as MariaDB or MySQL. You should not use the default SQLite database in production. - You should make sure to configure a database such as MariaDB or MySQL. You should not use the default SQLite database in production.
- Instead of `python manage.py runserver`, you should use a production-ready web server such as Caddy and gunicorn. - Instead of `quackscape-manage runserver`, you should use a production-ready web server such as Caddy and gunicorn.
- Instead of `python manage.py runserver`, you will want to use the `python manage.py runworker` command to start the background worker process. - Instead of `quackscape-manage runserver`, you will want to use the `quackscape-manage runworker` command to start the background worker process.
- You may want to use systemd or another process manager to keep the server and worker processes running in the background. - You may want to use systemd or another process manager to keep the server and worker processes running in the background.
## S3 Storage
Quackscape supports storing your static files and uploaded media in S3 buckets. During development, only MinIO was tested, but other providers, such as AWS, should work as well.
This has the advantage that your local hard drive needs less capacity, but note that the workers that do your video processing will still need to have enough disk space for this task.
Using S3 storage also allows you to run multiple public-facing instances, for example to balance the load, while ensuring to serve the same statics and media content on all instances.
To enable S3, refer to the provided section in `settings.dist.ini`. After that, you can run `quackscape-manage collectstatic` to upload your static files to S3.
One feature that may be of particular interest to those who want to work on the frontend is the `LocalStatic` option. If this is set to `1`, static files will be served by the local server instead of S3, which ensures you always work with the current state without having to run `collectstatic` all the time. In conjunction with `npm run watch:dev`, this makes it easy to work on the frontend without having to worry about the static files, while still being able to use S3 to store the media files.
## Workers ## Workers
Quackscape uses a background worker to process uploaded photos and videos. You can start the worker process using the `python manage.py runworker` command. Quackscape uses a background worker to process uploaded photos and videos. You can start the worker process using the `quackscape-manage runworker` command.
You may want to run the worker process on another machine. The server part is not very resource-hungry, so it can easily run on a VPS, but for video processing, you may want to use a machine with more resources, ideally with a powerful GPU. You may want to run the worker process on another machine. The server part is not very resource-hungry, so it can easily run on a VPS, but for video processing, you may want to use a machine with more resources, ideally with a powerful GPU.
@ -84,6 +108,10 @@ Redis = redis://<redis-ip>:6379/0
Replace `<redis-ip>` with the IP address of the machine running the Redis server. This way, the worker will be able to fetch tasks from the same Redis server the Quackscape server is writing them to. Replace `<redis-ip>` with the IP address of the machine running the Redis server. This way, the worker will be able to fetch tasks from the same Redis server the Quackscape server is writing them to.
## Contributing
We welcome contributions to this project. Please see the [CONTRIBUTING](CONTRIBUTING.md) file for more information.
## License ## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

48
pyproject.toml Normal file
View file

@ -0,0 +1,48 @@
[tool.poetry]
name = "quackscape"
version = "0.1.0"
description = ""
authors = ["Private.coffee Team <support@private.coffee>"]
license = "MIT"
readme = "README.md"
homepage = "https://quackscape.io"
repository = "https://git.private.coffee/PrivateCoffee/quackscape"
[tool.poetry.dependencies]
python = "^3.8"
django = "*"
djangorestframework = "*"
django-storages = "*"
django-polymorphic = "*"
setuptools = "*"
pillow = "*"
pygments = "*"
markdown = "*"
coreapi = "*"
pyyaml = "*"
django-autosecretkey = "*"
celery = "*"
redis = "*"
django-celery-results = "*"
django-celery-beat = "*"
drf-spectacular = {extras = ["sidecar"], version = "*"}
boto3 = "*"
argon2-cffi = "*"
django-csp = "*"
django-rest-polymorphic = "*"
[tool.poetry.group.mysql.dependencies]
mysqlclient = "*"
[tool.poetry.group.postgres.dependencies]
psycopg2 = "*"
[tool.poetry.dev-dependencies]
pytest = "^5.2"
[tool.poetry.scripts]
quackscape-manage = "quackscape.manage:main"
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

View file

@ -1,22 +0,0 @@
django
djangorestframework
django-storages
django-polymorphic
mysqlclient
setuptools
pillow
pygments
markdown
coreapi
pyyaml
django-autosecretkey
celery
redis
django-celery-results
django-celery-beat
drf-spectacular[sidecar]
boto3
psycopg2
argon2-cffi
django-csp
django-rest-polymorphic