LI
LinYuMingBejing/booking.com
使用Flask、MongoDB、Redis、Celery與Docker,定時向Booking.com爬取住房資訊,同時撰寫相關API,進行住房查詢,並且導入Swagger進行API的測試,加入Celery Flower和Grafana監測隊列任務與API效能,以及使用Logstash蒐集Nginx日誌。
Booking.com
Enviroment:
- Ubuntu: 16.04
- Python: 3.6.2
- Backend: Flask
- Test: Swagger
- Database: MongoDB, Redis
- Asynchronous: Celery (Redis)
- Monitoring Tool: Supervisor, Prometheus, Grafana
- Collecting Logs: Logstash
Install Docker
$ sudo apt-get update
$ sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo apt-key fingerprint 0EBFCD88
$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu
$(lsb_release -cs)
stable"
$ sudo apt-get update
$ sudo apt-get install docker-ce docker-ce-cli containerd.io
Install Docker Compose
$ sudo apt install docker-compose
Deploy Booking.com Application
- Deploy
$ cd ./docker_stg
$ sudo docker-compose up --build -d
- Check docker process
$ sudo docker ps
- Get docker log
$ sudo docker-compose logs --tail=20 -f flask
Swagger
Monitor process
$ supervisorctl
Monitor uwsgi application
uwsgitop 127.0.0.1:5002
Monitor celery worker
Prometheus Monitor
Grafana Monitor
- http://localhost:3000
- Username: admin
- Password: pass
Nginx Log Monitor
Note1: How to allow certain ips connect mongodb?
- vim /etc/mongod.conf
# bindIp: 127.0.0.1
bindIp: 0.0.0.0
- restart mongodb
$ sudo service mongod restart
$ sudo service mongod status
- firewall settings
$ sudo ufw enable
$ sudo ufw deny from 192.168.18.0/24 to any port 27017
$ sudo ufw allow from 192.168.18.0/24 to any port 27017
$ sudo ufw status
Note2: How to create multiple user on mongodb?
- create user
> use booking;
switched to db booking
> db.createUser(
{
user: "dbadmin",
pwd: "StrongPassword",
roles: [ { role: "readWrite", db: "booking" } ]
}
)
> exit
bye
- vim /etc/mongod.conf
security:
authorization: enabled
- restart mongodb
$ sudo service mongod restart
$ sudo service mongod status
Reference

