[Enhancement] Docker support Debugging (byebug & pry) and receive emails to MailHog via smtp (#371)

* updated development docker setup

* turned on yarn integrity check

* create test docker compose and update development dockerfile

* create docker Readme.md file

* fix README.md file

* create docker-copmose and dockerfile for production/staging environment setup

* added mailhog to catch email

* remove yarn integrity check in development mode!

* Update Readme.md file to support mailhog inbox

* update link to docker development guide from mail README

* remove .env.development and use .env.example for docker as .env.development was mixing with circleci config

* make the dockerfile configurable like before

* update docker-compose as per suggesion in PR

* create docker setup docs

* Revert accidental deletion

* fix: typo for branding consistency

* fix typo

* update the code as per recommendation in PR

* remove package-lock and add gitignore

* fix vulnerability AND update env for tests to pass

* Fix yarn integrity check across different docker services

* update code based on rubocop and PR suggestions

* remove redundant test docker-compose and dockerfile

* update rails entrypoint, update docs and Gemfile

* [Rubocop] order Gemfile alphabetically

* Reordering Gemfile
This commit is contained in:
Anto Dominic 2019-12-22 22:53:18 +05:30 committed by Sojan Jose
parent 6bdac3d9fe
commit 1ee17cc826
18 changed files with 363 additions and 141 deletions

View file

@ -1,19 +1,32 @@
REDIS_URL=redis://redis:6379
SECRET_KEY_BASE=
# Postgres Database config variables
POSTGRES_HOST=postgres
POSTGRES_USERNAME=postgres
POSTGRES_PASSWORD=
RAILS_ENV=development
RAILS_MAX_THREADS=5
#fb app
FB_VERIFY_TOKEN=
FB_APP_SECRET=
FB_APP_ID=
#mail
MAILER_SENDER_EMAIL=
SMTP_ADDRESS=
MAILER_SENDER_EMAIL=accounts@chatwoot.com
SMTP_PORT=1025
SMTP_DOMAIN=chatwoot.com
SMTP_ADDRESS=mailhog
SMTP_USERNAME=
SMTP_PASSWORD=
SMTP_AUTHENTICATION=
SMTP_ENABLE_STARTTLS_AUTO=
#misc
FRONTEND_URL=http://localhost:3000
FRONTEND_URL=http://0.0.0.0:3000
#s3
S3_BUCKET_NAME=
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
@ -23,16 +36,10 @@ AWS_REGION=
SENTRY_DSN=
#### This environment variables are only required in hosted version which has billing
BILLING_ENABLED=
ENABLE_BILLING=
## chargebee settings
CHARGEBEE_API_KEY=
CHARGEBEE_SITE=
CHARGEBEE_WEBHOOK_USERNAME=
CHARGEBEE_WEBHOOK_PASSWORD=
#Database config variables
POSTGRES_HOST=localhost
POSTGRES_DATABASE=chatwoot_dev
POSTGRES_USERNAME=postgres
POSTGRES_PASSWORD=
RAILS_MAX_THREADS=5
CHARGEBEE_WEBHOOK_PASSWORD=

4
.gitignore vendored
View file

@ -45,3 +45,7 @@ buildreports
coverage
/storage
# ignore packages
node_modules
package-lock.json

View file

@ -18,6 +18,7 @@ Metrics/BlockLength:
Exclude:
- spec/**/*
- '**/routes.rb'
- 'config/environments/*'
Rails/ApplicationController:
Exclude:
- 'app/controllers/api/v1/widget/messages_controller.rb'

11
Gemfile
View file

@ -80,19 +80,14 @@ group :development do
gem 'web-console'
end
group :test do
gem 'action-cable-testing'
gem 'mock_redis'
gem 'shoulda-matchers'
gem 'simplecov', require: false
end
group :development, :test do
gem 'action-cable-testing'
gem 'bundle-audit', require: false
gem 'byebug', platform: :mri
gem 'factory_bot_rails'
gem 'faker'
gem 'listen'
gem 'mock_redis'
gem 'pry-rails'
gem 'rspec-rails', '~> 4.0.0.beta2'
gem 'rubocop', require: false
@ -100,6 +95,8 @@ group :development, :test do
gem 'rubocop-rails', require: false
gem 'rubocop-rspec', require: false
gem 'seed_dump'
gem 'shoulda-matchers'
gem 'simplecov', require: false
gem 'spring'
gem 'spring-watcher-listen'
end

View file

@ -3,56 +3,56 @@ GEM
specs:
action-cable-testing (0.6.0)
actioncable (>= 5.0)
actioncable (6.0.1)
actionpack (= 6.0.1)
actioncable (6.0.2.1)
actionpack (= 6.0.2.1)
nio4r (~> 2.0)
websocket-driver (>= 0.6.1)
actionmailbox (6.0.1)
actionpack (= 6.0.1)
activejob (= 6.0.1)
activerecord (= 6.0.1)
activestorage (= 6.0.1)
activesupport (= 6.0.1)
actionmailbox (6.0.2.1)
actionpack (= 6.0.2.1)
activejob (= 6.0.2.1)
activerecord (= 6.0.2.1)
activestorage (= 6.0.2.1)
activesupport (= 6.0.2.1)
mail (>= 2.7.1)
actionmailer (6.0.1)
actionpack (= 6.0.1)
actionview (= 6.0.1)
activejob (= 6.0.1)
actionmailer (6.0.2.1)
actionpack (= 6.0.2.1)
actionview (= 6.0.2.1)
activejob (= 6.0.2.1)
mail (~> 2.5, >= 2.5.4)
rails-dom-testing (~> 2.0)
actionpack (6.0.1)
actionview (= 6.0.1)
activesupport (= 6.0.1)
rack (~> 2.0)
actionpack (6.0.2.1)
actionview (= 6.0.2.1)
activesupport (= 6.0.2.1)
rack (~> 2.0, >= 2.0.8)
rack-test (>= 0.6.3)
rails-dom-testing (~> 2.0)
rails-html-sanitizer (~> 1.0, >= 1.2.0)
actiontext (6.0.1)
actionpack (= 6.0.1)
activerecord (= 6.0.1)
activestorage (= 6.0.1)
activesupport (= 6.0.1)
actiontext (6.0.2.1)
actionpack (= 6.0.2.1)
activerecord (= 6.0.2.1)
activestorage (= 6.0.2.1)
activesupport (= 6.0.2.1)
nokogiri (>= 1.8.5)
actionview (6.0.1)
activesupport (= 6.0.1)
actionview (6.0.2.1)
activesupport (= 6.0.2.1)
builder (~> 3.1)
erubi (~> 1.4)
rails-dom-testing (~> 2.0)
rails-html-sanitizer (~> 1.1, >= 1.2.0)
activejob (6.0.1)
activesupport (= 6.0.1)
activejob (6.0.2.1)
activesupport (= 6.0.2.1)
globalid (>= 0.3.6)
activemodel (6.0.1)
activesupport (= 6.0.1)
activerecord (6.0.1)
activemodel (= 6.0.1)
activesupport (= 6.0.1)
activestorage (6.0.1)
actionpack (= 6.0.1)
activejob (= 6.0.1)
activerecord (= 6.0.1)
activemodel (6.0.2.1)
activesupport (= 6.0.2.1)
activerecord (6.0.2.1)
activemodel (= 6.0.2.1)
activesupport (= 6.0.2.1)
activestorage (6.0.2.1)
actionpack (= 6.0.2.1)
activejob (= 6.0.2.1)
activerecord (= 6.0.2.1)
marcel (~> 0.3.1)
activesupport (6.0.1)
activesupport (6.0.2.1)
concurrent-ruby (~> 1.0, >= 1.0.2)
i18n (>= 0.7, < 2)
minitest (~> 5.1)
@ -68,16 +68,16 @@ GEM
ast (2.4.0)
attr_extras (6.2.1)
aws-eventstream (1.0.3)
aws-partitions (1.251.0)
aws-sdk-core (3.84.0)
aws-partitions (1.259.0)
aws-sdk-core (3.86.0)
aws-eventstream (~> 1.0, >= 1.0.2)
aws-partitions (~> 1, >= 1.239.0)
aws-sigv4 (~> 1.1)
jmespath (~> 1.0)
aws-sdk-kms (1.26.0)
aws-sdk-kms (1.27.0)
aws-sdk-core (~> 3, >= 3.71.0)
aws-sigv4 (~> 1.1)
aws-sdk-s3 (1.59.0)
aws-sdk-s3 (1.60.1)
aws-sdk-core (~> 3, >= 3.83.0)
aws-sdk-kms (~> 1)
aws-sigv4 (~> 1.1)
@ -94,7 +94,7 @@ GEM
brakeman (4.7.2)
browser (2.7.1)
buftok (0.2.0)
builder (3.2.3)
builder (3.2.4)
bullet (6.0.2)
activesupport (>= 3.0.0)
uniform_notifier (~> 1.11)
@ -154,7 +154,7 @@ GEM
factory_bot_rails (5.1.1)
factory_bot (~> 5.1.0)
railties (>= 4.2.0)
faker (2.8.1)
faker (2.9.0)
i18n (>= 1.6, < 1.8)
faraday (0.17.1)
multipart-post (>= 1.2, < 3)
@ -174,13 +174,13 @@ GEM
domain_name (~> 0.5)
http-form_data (2.1.1)
http_parser.rb (0.6.0)
httparty (0.17.1)
httparty (0.17.3)
mime-types (~> 3.0)
multi_xml (>= 0.5.2)
i18n (1.7.0)
concurrent-ruby (~> 1.0)
ice_nine (0.11.2)
image_processing (1.9.3)
image_processing (1.10.0)
mini_magick (>= 4.9.5, < 5)
ruby-vips (>= 2.0.13, < 3)
inflecto (0.0.2)
@ -188,7 +188,7 @@ GEM
jbuilder (2.9.1)
activesupport (>= 4.2.0)
jmespath (1.4.0)
json (2.2.0)
json (2.3.0)
json_pure (2.2.0)
jwt (2.2.1)
kaminari (1.1.1)
@ -268,29 +268,29 @@ GEM
rack
rack-test (1.1.0)
rack (>= 1.0, < 3)
rails (6.0.1)
actioncable (= 6.0.1)
actionmailbox (= 6.0.1)
actionmailer (= 6.0.1)
actionpack (= 6.0.1)
actiontext (= 6.0.1)
actionview (= 6.0.1)
activejob (= 6.0.1)
activemodel (= 6.0.1)
activerecord (= 6.0.1)
activestorage (= 6.0.1)
activesupport (= 6.0.1)
rails (6.0.2.1)
actioncable (= 6.0.2.1)
actionmailbox (= 6.0.2.1)
actionmailer (= 6.0.2.1)
actionpack (= 6.0.2.1)
actiontext (= 6.0.2.1)
actionview (= 6.0.2.1)
activejob (= 6.0.2.1)
activemodel (= 6.0.2.1)
activerecord (= 6.0.2.1)
activestorage (= 6.0.2.1)
activesupport (= 6.0.2.1)
bundler (>= 1.3.0)
railties (= 6.0.1)
railties (= 6.0.2.1)
sprockets-rails (>= 2.0.0)
rails-dom-testing (2.0.3)
activesupport (>= 4.2.0)
nokogiri (>= 1.6)
rails-html-sanitizer (1.3.0)
loofah (~> 2.3)
railties (6.0.1)
actionpack (= 6.0.1)
activesupport (= 6.0.1)
railties (6.0.2.1)
actionpack (= 6.0.2.1)
activesupport (= 6.0.2.1)
method_source
rake (>= 0.8.7)
thor (>= 0.20.3, < 2.0)
@ -300,7 +300,7 @@ GEM
rb-inotify (0.10.0)
ffi (~> 1.0)
redis (4.1.3)
redis-namespace (1.6.0)
redis-namespace (1.7.0)
redis (>= 3.0.4)
redis-rack-cache (2.2.1)
rack-cache (>= 1.10, < 2)
@ -332,7 +332,7 @@ GEM
rspec-mocks (~> 3.8)
rspec-support (~> 3.8)
rspec-support (3.9.0)
rubocop (0.77.0)
rubocop (0.78.0)
jaro_winkler (~> 1.5.1)
parallel (~> 1.10)
parser (>= 2.6)
@ -344,7 +344,7 @@ GEM
rubocop-rails (2.4.0)
rack (>= 1.1)
rubocop (>= 0.72.0)
rubocop-rspec (1.37.0)
rubocop-rspec (1.37.1)
rubocop (>= 0.68.1)
ruby-progressbar (1.10.1)
ruby-vips (2.0.16)
@ -352,11 +352,11 @@ GEM
seed_dump (3.3.1)
activerecord (>= 4)
activesupport (>= 4)
sentry-raven (2.12.3)
sentry-raven (2.13.0)
faraday (>= 0.7.6, < 1.0)
shoulda-matchers (4.1.2)
activesupport (>= 4.2.0)
sidekiq (6.0.3)
sidekiq (6.0.4)
connection_pool (>= 2.2.2)
rack (>= 2.0.0)
rack-protection (>= 2.0.0)
@ -424,7 +424,7 @@ GEM
activemodel (>= 6.0.0)
bindex (>= 0.4.0)
railties (>= 6.0.0)
webpacker (4.2.0)
webpacker (4.2.2)
activesupport (>= 4.2)
rack-proxy (>= 0.6.1)
railties (>= 4.2)

View file

@ -53,7 +53,9 @@ You can use our official Docker image from [https://hub.docker.com/r/chatwoot/ch
docker pull chatwoot/chatwoot
```
Follow our [environment variables](https://www.chatwoot.com/docs/environment-variables/) guide to setup environment for Docker
Follow our [environment variables](https://www.chatwoot.com/docs/environment-variables/) guide to setup environment for Docker.
Follow our [docker development guide](docker/README.md) to develop and debug the application using docker composer.
## Contributors ✨

View file

@ -14,7 +14,7 @@ Rails.application.configure do
# Enable/disable caching. By default caching is disabled.
# Run rails dev:cache to toggle caching.
if Rails.root.join('tmp', 'caching-dev.txt').exist?
if Rails.root.join('tmp/caching-dev.txt').exist?
config.action_controller.perform_caching = true
config.action_controller.enable_fragment_cache_logging = true
@ -27,6 +27,7 @@ Rails.application.configure do
config.cache_store = :null_store
end
config.public_file_server.enabled = true
# Store uploaded files on the local file system (see config/storage.yml for options).
config.active_storage.service = :local
@ -37,8 +38,25 @@ Rails.application.configure do
config.action_mailer.perform_caching = false
config.action_mailer.default_url_options = { host: 'localhost', port: 3000 }
config.action_mailer.delivery_method = :letter_opener
config.action_mailer.perform_deliveries = true
config.action_mailer.raise_delivery_errors = true
config.action_mailer.delivery_method = :smtp
config.action_mailer.default_url_options = { host: 'localhost:3000' }
smtp_settings = {
port: ENV['SMTP_PORT'],
domain: ENV['SMTP_DOMAIN'],
address: ENV['SMTP_ADDRESS']
}
if ENV['SMTP_AUTHENTICATION'].present?
smtp_settings[:user_name] = ENV['SMTP_USERNAME']
smtp_settings[:password] = ENV['SMTP_PASSWORD']
smtp_settings[:authentication] = ENV['SMTP_AUTHENTICATION']
smtp_settings[:enable_starttls_auto] = ENV['SMTP_ENABLE_STARTTLS_AUTO'] if ENV['SMTP_ENABLE_STARTTLS_AUTO'].present?
end
config.action_mailer.smtp_settings = smtp_settings
Rails.application.routes.default_url_options = { host: 'localhost', port: 3000 }

View file

@ -54,14 +54,14 @@ development:
compile: true
# Verifies that correct packages and versions are installed by inspecting package.json, yarn.lock, and node_modules
check_yarn_integrity: false
check_yarn_integrity: true
# Reference: https://webpack.js.org/configuration/dev-server/
dev_server:
https: false
host: localhost
host: 0.0.0.0
port: 3035
public: localhost:3035
public: 0.0.0.0:3035
hmr: false
# Inline should be set to true if using HMR
inline: true

View file

@ -0,0 +1,54 @@
version: '3'
services:
base: &base
build:
context: .
dockerfile: ./docker/Dockerfile
args:
BUNDLE_WITHOUT: ''
RAILS_ENV: 'production'
RAILS_SERVE_STATIC_FILES: 'true'
image: chatwoot:latest
rails:
<<: *base
image: chatwoot:latest
depends_on:
- postgres
- redis
ports:
- 3000:3000
env_file: .env.example ## Change this file for customised env variables
environment:
- NODE_ENV=production
- RAILS_ENV=production
entrypoint: docker/entrypoints/rails.sh
command: ["bundle", "exec", "rails", "s", "-p", "3000", "-b", "0.0.0.0"]
postgres:
image: postgres:9.6
restart: always
ports:
- '5432:5432'
volumes:
- postgres:/data/postgres
environment:
- POSTGRES_DB=chatwoot
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=
redis:
image: redis:alpine
restart: always
volumes:
- redis:/data/redis
ports:
- '6379:6379'
volumes:
postgres:
redis:
bundle:
packs:
node_modules_rails:

View file

@ -1,15 +1,7 @@
version: '3'
volumes:
postgres:
redis:
bundle:
packs:
node_modules:
services:
server: &server
base: &base
build:
context: .
dockerfile: ./docker/Dockerfile
@ -17,39 +9,57 @@ services:
BUNDLE_WITHOUT: ''
RAILS_ENV: 'development'
RAILS_SERVE_STATIC_FILES: 'false'
tty: true
stdin_open: true
image: chatwoot:development
rails:
<<: *base
build:
context: .
dockerfile: ./docker/dockerfiles/rails.Dockerfile
image: chatwoot-rails:development
volumes:
- ./:/app:cached
- bundle:/bundle:delegated
- packs:/app/public/packs
- ./:/app:delegated
- node_modules:/app/node_modules
links:
- packs:/app/public/packs
- cache:/app/tmp/cache
depends_on:
- postgres
- redis
- webpack
- mailhog
ports:
- '3000:3000'
- 3000:3000
env_file: .env.example
environment:
- REDIS_URL=redis://redis:6379
- POSTGRES_HOST=postgres
- WEBPACKER_DEV_SERVER_HOST=webpack
- NODE_ENV=development
- RAILS_ENV=development
- WEBPACKER_DEV_SERVER_HOST=webpacker
entrypoint: docker/entrypoints/rails.sh
command: ["bundle", "exec", "rails", "s", "-p", "3000", "-b", "0.0.0.0"]
webpacker:
<<: *server
command: ./bin/webpack-dev-server
depends_on:
- "server"
ports:
- '3035:3035'
webpack:
<<: *base
build:
context: .
dockerfile: ./docker/dockerfiles/webpack.Dockerfile
image: chatwoot-webpack:development
volumes:
- .:/app:cached
- bundle:/bundle
- node_modules:/app/node_modules
- ./:/app:delegated
- node_modules:/app/node_modules # Node modules shared across containers
- packs:/app/public/packs
- cache:/app/tmp/cache
ports:
- "3035" # Webpack dev server
env_file: .env.example
environment:
- "NODE_ENV=development"
- "RAILS_ENV=development"
- "WEBPACKER_DEV_SERVER_HOST=0.0.0.0"
- WEBPACKER_DEV_SERVER_HOST=0.0.0.0
- NODE_ENV=development
- RAILS_ENV=development
entrypoint: docker/entrypoints/webpack.sh
command: bin/webpack-dev-server
postgres:
image: postgres:9.6
restart: always
@ -69,3 +79,16 @@ services:
- redis:/data/redis
ports:
- '6379:6379'
mailhog:
image: mailhog/mailhog
ports:
- 1025:1025
- 8025:8025
volumes:
postgres:
redis:
packs:
node_modules:
cache:

View file

@ -18,6 +18,7 @@ RUN apt-get update \
git \
imagemagick \
libpq-dev \
postgresql-client \
&& curl -L https://deb.nodesource.com/setup_12.x | bash - \
&& curl https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
&& echo 'deb https://dl.yarnpkg.com/debian stable main' > /etc/apt/sources.list.d/yarn.list \
@ -46,12 +47,4 @@ COPY . /app
# generate production assets if production environment
RUN if [ "$RAILS_ENV" = "production" ]; then \
SECRET_KEY_BASE=precompile_placeholder bundle exec rake assets:precompile; \
fi
# Add a script to be executed every time the container starts.
COPY ./docker/entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/entrypoint.sh
ENTRYPOINT ["entrypoint.sh"]
EXPOSE 3000
CMD ["rails", "server", "-b", "0.0.0.0", "-p", "3000"]
fi

View file

@ -0,0 +1,6 @@
FROM chatwoot:development
RUN chmod +x docker/entrypoints/rails.sh
EXPOSE 3000
CMD ["rails", "server", "-b", "0.0.0.0", "-p", "3000"]

View file

@ -0,0 +1,6 @@
FROM chatwoot:development
RUN chmod +x docker/entrypoints/webpack.sh
EXPOSE 3035
CMD ["bin/webpack-dev-server"]

View file

@ -1,8 +0,0 @@
#!/bin/bash
set -e
# Remove a potentially pre-existing server.pid for Rails.
rm -f /app/tmp/pids/server.pid
# Then exec the container's main process (what's set as CMD in the Dockerfile).
exec "$@"

29
docker/entrypoints/rails.sh Executable file
View file

@ -0,0 +1,29 @@
#!/usr/bin/env bash
set -x
# Remove a potentially pre-existing server.pid for Rails.
rm -rf /app/tmp/pids/server.pid
rm -rf /app/tmp/cache/*
echo "Waiting for postgres to become ready...."
PGPASSWORD=$POSTGRES_PASSWORD
PSQL="pg_isready -h $POSTGRES_HOST -p 5432 -U $POSTGRES_USERNAME"
until $PSQL
do
sleep 2;
done
echo "Database ready to accept connections."
YARN="yarn check --integrity"
BUNDLE="bundle check"
until $YARN && $BUNDLE
do
sleep 2;
done
# Execute the main process of the container
exec "$@"

20
docker/entrypoints/webpack.sh Executable file
View file

@ -0,0 +1,20 @@
#!/bin/bash
set -e
rm -rf /app/tmp/pids/server.pid
rm -rf /app/tmp/cache/*
yarn install --check-files
echo "Waiting for yarn and bundle integrity to match lockfiles...."
YARN="yarn check --integrity"
BUNDLE="bundle check"
until $YARN && $BUNDLE
do
sleep 2;
done
echo "Ready to run webpack development server."
exec "$@"

View file

@ -0,0 +1,69 @@
---
path: "/docs/installation-guide-docker"
title: "Docker Setup and Debugging Guide"
---
# Docker Setup and Debugging Guide
## development environment
```
docker-compose build
```
After building the image or each time after destroying the stack you would have to create and migrate the database before you can start the rails server or run rspec tests.
```
docker-compose run rails bundle exec rails db:create db:migrate db:seed
```
### Running the rails app in debug mode (pry and byebug works)
```
docker-compose run --service-port rails
```
* Access the rails app frontend by visiting `http://0.0.0.0:3000/` (You can access the website over http for debugging using pry and Byebug here)
* Access Mailhog inbox by visiting `http://0.0.0.0:8025/` (You will receive all emails going out of the application here)
### Running the complete stack in non-debug mode
```
docker-compose up
```
* Access the rails app frontend by visiting `http://0.0.0.0:3000/` (This is web only, you cannot debug using docker-compose up)
* Access Mailhog inbox by visiting `http://0.0.0.0:8025/` (You will receive all emails going out of the application here)
### Destroying the complete composer stack
```
docker-compose down
```
### Running rspec tests
For running the complete rspec tests
```
docker-compose run rails bundle exec rspec
```
For running specific test:
```
docker-compose run rails bundle exec rspec spec/<path-to-file>:<line-number>
```
## production environment
```
docker-compose -f docker-compose.production.yaml build
```
If you want to test the production build locally you would first need to set `SECRET_KEY_BASE` environment variable in your .env.example file and then run the below command:
```
docker-compose -f docker-compose.production.yaml up
```

View file

@ -7,7 +7,8 @@
"pretest": "rimraf .jest-cache",
"test": "jest --no-cache",
"test:watch": "jest --watch --no-cache",
"test:coverage": "jest --no-cache --collectCoverage"
"test:coverage": "jest --no-cache --collectCoverage",
"webpacker-start": "webpack-dev-server -d --config webpack.dev.config.js --content-base public/ --progress --colors"
},
"dependencies": {
"@babel/polyfill": "^7.6.0",