Z's Blog

Deploy a self-hosted MEMOS note system

部署一个自托管的MEMOS笔记系统
Posted on In , Views:
Symbols count in article: 7k Reading time ≈ 18 mins.

For personal knowledge management, I am very keen on creating my own local-first and open source self-hosted services. This way, I can avoid relying on any platform, have complete autonomy over my data, and be able to migrate services at any time. Options like Obsidian, Hexo blog, and the memos introduced in this article all meet these criteria.

The protagonist of this article, memos, is an open-source lightweight note-taking service that allows you to take notes in a way similar to posting on Weibo, supporting TAG marking and citations. It features an account system and permission management, allowing web access at any time, and notes can be public or private, offering great flexibility. I chose it because it complements Obsidian; Obsidian is still too heavy for my needs—it works well on PC but has a poor mobile experience. Thus, I need a lightweight, always-available note-taking service.

This article will introduce how to deploy a Memos service on a VPS using Docker, along with Nginx to bind the domain name, Certbot to issue and automatically update SSL certificates, and regular backups of the memos database, as well as some optimization configurations I made for the service.

Prerequisites

Before proceeding with this article, you need to meet the following conditions:

  1. You must have a VPS (preferably overseas, without needing to file the domain).
  2. A domain name (the article uses memos.example.com as an example).
  3. Familiarity with the Linux system and common commands.

Deployment

Docker

First, you need to install Docker on the VPS:

1
2
apt-get install docker
apt-get install docker-compose

Common Docker commands:

1
2
3
4
5
6
7
8
9
10
11
12
13
# List all images
docker images -a
# Delete an image
docker rmi IMAGE_ID

# List all containers
docker ps -a
# Stop a container
docker stop CONTAINER_ID
# Start a container
docker start CONTAINER_ID
# Remove a container
docker rm CONTAINER_ID

Memos

Then, use Docker to install and run the memos image. The latest version is 0.21.0:

1
docker run -d --name memos -p 5230:5230 -v ~/.memos/:/var/opt/memos neosmemo/memos:stable

-p specifies the port, and ~/.memos/:/var/opt/memos specifies the data storage path for memos. By default, it is placed in the ~/.memos/ directory, but it can be changed as needed.

After executing, the following files will be generated:

The memo_prod.db file is the Sqlite database for memos and can be opened with tools like DB Browser for Sqlite to access the data.

Once the above Docker command has been executed, the memos service is already running, and you can access it via localhost:5230.

Container Auto-Start

First, check the Docker ID:

1
2
3
root@vultr:~# docker ps -a
CONTAINER ID   IMAGE                   COMMAND     CREATED       STATUS       PORTS                                       NAMES
3f25b36e833e   neosmemo/memos:stable   "./memos"   4 hours ago   Up 4 hours   0.0.0.0:5230->5230/tcp, :::5230->5230/tcp   memos

Then set it to restart automatically:

1
docker update --restart=always 3f25b36e833e

To turn off auto-start:

1
docker update --restart=no

Custom Domain Name

After executing the above Docker command, a web service has been started locally. However, it can only be accessed via IP. We can use Nginx to map it to a domain name.

First, install Nginx:

1
apt-get install nginx

Check the default Nginx configuration file:

1
2
3
nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Modify /etc/nginx/nginx.conf:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
        worker_connections 768;
        # multi_accept on;
}

http {
        ##
        # Basic Settings
        ##
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;
        # server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;

        ##
        # Logging Settings
        ##
        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        ##
        # Gzip Settings
        ##
        gzip on;
        # gzip_vary on;
        # gzip_proxied any;
        # gzip_comp_level 6;
        # gzip_buffers 16 8k;
        # gzip_http_version 1.1;
        # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

        ##
        # Virtual Host Configs
        ##
        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}

Then, you can create the Nginx configuration for memos in the /etc/nginx/conf.d/ directory:

1
2
3
4
5
6
7
8
9
10
11
12
# memos.example.com.conf
server {
    server_name memos.example.com;

    location / {
        proxy_pass http://localhost:5230;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

This only adds the http mapping, and HTTPS cannot be opened yet. You still need to apply for an SSL certificate and then configure an Nginx configuration for 443, which will be introduced next.

SSL Certificate

Certbot

Before applying for the certificate, please ensure that the DNS is already resolved to the machine’s IP.

Install certbot:

1
apt-get install certbot

Then issue the certificate:

1
certbot certonly --standalone -d example.com

Note: When executing the command, make sure that Nginx and memos services are not running to ensure they are shut down. You can stop the Nginx service with service nginx stop and start it with service nginx start.

Once the certificate is issued, it will be stored in the /etc/letsencrypt/live/ directory:

At this point, the certificate has been successfully applied.

Then you need to edit the previously created Nginx configuration file (in the /etc/nginx/conf.d/ directory) and create a 443 ssl configuration to enable HTTPS:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# memos.example.com.conf
server {
    listen 443 ssl;
    server_name memos.example.com;

    client_max_body_size 1024m;

    ssl_certificate      /etc/letsencrypt/live/memos.example.com/fullchain.pem;
    ssl_certificate_key  /etc/letsencrypt/live/memos.example.com/privkey.pem;

    ssl_session_timeout  5m;

    ssl_ciphers  HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers  on;

    location / {
        proxy_pass http://localhost:5230;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Now, our domain supports HTTPS access.

Automate Certificate Renewal

The certificates issued by Certbot are valid for only three months and need to be reapplied. To automate this process, you can set a crontab task:

1
crontab -e

Add the following command:

1
0 3 1 * * certbot renew --quiet --pre-hook "service nginx stop" --post-hook "service nginx start"

This means that on the first day of every month at 3 AM, the renewal task will be executed, shutting down the Nginx service before execution and restarting it afterward.

Troubleshooting

If the certbot certificate application fails, be sure to check the local port occupancy to ensure that the Nginx service has exited and that no other program is using ports 80 and 443. If you’re unsure whether a port is occupied, you can use lsof to check:

1
2
3
4
5
6
root@vultr:~# lsof -i :80
COMMAND     PID     USER   FD   TYPE    DEVICE SIZE/OFF NODE NAME
nginx   1178482     root    6u  IPv4 121041008      0t0  TCP *:http (LISTEN)
nginx   1178482     root    8u  IPv6 121041010      0t0  TCP *:http (LISTEN)
nginx   1178483 www-data    6u  IPv4 121041008      0t0  TCP *:http (LISTEN)
nginx   1178483 www-data    8u  IPv6 121041010      0t0  TCP *:http (LISTEN)

If you encounter the following error while using certbot renew:

You can try the following commands:

1
2
3
sudo apt-get update
sudo apt-get install --only-upgrade certbot
pip install --upgrade certbot urllib3 requests

Database

Automated Backups

Similarly, we can set up regular backups for our MEMOS database. Data security should always be a priority.

You can still use crontab for automated backups, submitting them in a GIT repository format.

  1. First, create a private repository on GitHub.
  2. On the VPS, navigate to the memos data directory.
  3. Create a git repository and make an initial commit.
  4. Create a crontab scheduled task.

Regarding backup frequency, I currently set it to run once every 5 minutes, but you can adjust the frequency as needed.

Additionally, it is important to note that you should not directly use git to manage a database that is currently running, as this may cause database corruption. You should back it up first and then manage that backup with git.

I configured the following .gitignore to ignore all files under ./memos, only tracking files starting with backup*:

1
2
.memos/*
!.memos/backup*

The command for backing up the database is:

1
sqlite3 memos_prod.db ".backup backup_memos_prod.db"

The complete crontab command is as follows:

1
*/5 * * * * cd /root/.memos/ && /usr/bin/sqlite3 memos_prod.db ".backup backup_memos_prod.db" && /usr/bin/git add . && /usr/bin/git commit -m 'update' && /usr/bin/git push origin master

Note: The commands in crontab should use absolute paths; relative paths like ~ may not work correctly.

Database Repair

If you receive the following error when accessing memos:

1
database disk image is malformed

This indicates that the database is corrupted, and we need to migrate the data out and create a new database.

1
2
3
4
5
6
7
# install sqlite3
apt-get install sqlite3

# Dump the database to SQL
sqlite3 memos_prod.db ".dump" > memos_prod.sql
# Create a new database using SQL
sqlite new_memos_prod.db < memos_prod.sql

Then move or delete the old db, rename the new one to memos_prod.db, and restart Docker.

Data Migration

Some of my earlier notes were stored in a separate md file alongside the blog, in formats like the following:

1
2
3
4
5
6
## 2024-04-08 17:18
AAA,BBBBBBBBB,CCCCC

## 2024-04-08 17:19
DDDDDD,EEEEEEEEEEE,FFFFFFFF
![](img.png)

After writing for years, I now have hundreds of entries. If I needed to manually transfer them to Memos, it would be quite cumbersome, and I wouldn’t want to do that.

Using GPT-4 combined with manually optimized code, I generated a Python script to convert md files into a CSV file that conforms to Memos database table standards. This way, I just need to manually import the CSV once to complete the data migration.

Script: md2memos.py.

The execution command will generate two CSV files in the md directory for import into the db:

1
python md2memos.py --md MARKDOWN_FILE.md

Note: When generating CSV files for the Memos database, be sure to import them separately into the memo and resource tables. Then perform the following operations on the resource table to ensure these two items are empty.

1
2
UPDATE resource SET blob = NULL;
UPDATE resource SET internal_path = '';

Configuration Optimization

Interface Beautification

  1. Change the font to LXGW WenKai.
  2. Integrate Bing’s daily wallpaper so that the background changes daily.
  3. Apply a semi-transparent effect.
  4. Hide some unnecessary options.

The CSS code is as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
/* Change font */
body{font-family: "LXGW WenKai Screen", sans-serif !important;}
/* Change Memo font size */
.memo-wrapper .text-base { font-size: 0.95rem}
/* Change code block font size */
.text-sm { font-size: 0.85rem; }
/* Hide Notification tab */
#header-inbox { display: none;}
/* Hide Profile tab */
#header-profile { display: none; }
/* Hide Explore tab */
/* #header-explore { display: none;} */
/* Hide About tab */
#header-about { display: none; }
/* Change editor font to monospace */
textarea { font-family: 'Courier New', Courier, monospace;}
/* Hide 'via memos' */
body .flex.flex-row.justify-between.items-center > .text-gray-500.dark:text-gray-400 { display: none;}
/* Share memos width */
.share-memo-dialog>.dialog-container { width: auto; }

/* Sidebar */
.w-56 { width: 12rem; }
/* Comment */
.pt-16 { padding-top: 2rem; }

blockquote{
    border: 1px solid #246ad1 !important;
    border-left: 4px solid #246ad1 !important;
    position:relative;
}
.blockquote-center{ background: none; }

/* #root>div:nth-child(1) */
body
{
    background-image: url('https://bing.immmmm.com/img/bing?region=zh-CN&type=image');
    background-position: buttom;
    backdrop-filter: blur(10px);
    background-size: contain;
}

#root main,#root header,#root aside {
    background-color: rgba(244 244 245 / 60%) !important;
    background: content-box !important;
    border-radius: 5px !important;
}

#root main,#root header,#root aside>div:nth-child(2),#root aside>div:nth-child(3)
{
    background-color: white;
    border-radius: 5px;
}

.px-2{
    background: content-box !important;
}
.border-r {
    border-right-width: 0px !important;
}

/* Mobile top bar */
.sm\:pt-2 {
    background: unset !important;
    --tw-backdrop-blur: auto !important
}
/* Top bar text */
/* .text-gray-700{
    color: snow !important;
}*/

/* Set scrollbar styles */
::-webkit-scrollbar {
  width: 5px !important;
  height: 5px !important;
}
/* Scrollbar track */
::-webkit-scrollbar-track {
  background: #eee !important;
}
/* Scrollbar thumb */
::-webkit-scrollbar-thumb {
  border-radius: 5px !important;
  background-color: #ccc !important;
}
::-webkit-scrollbar-thumb:hover {
  background-color: rgb(247, 149, 51) !important;
}

JavaScript code:

1
2
3
4
5
6
7
8
function changeFont() {
  const link = document.createElement("link");
  link.rel = "stylesheet";
  link.type = "text/css";
  link.href = "https://cdn.staticfile.org/lxgw-wenkai-screen-webfont/1.7.0/lxgwwenkaiscreen.css";
  document.head.append(link);
};
changeFont()

S3 Service

The configuration and published text of Memos are stored in an SQLite database, but SQLite is not friendly towards binary files. Moreover, managing these binary files within the database is not ideal. Therefore, you can configure an S3 service (such as Tencent Cloud COS) to handle uploaded files and images.

Item Configuration
Name TencentCOS
Endpoint https://cos.ap-guangzhou.myqcloud.com
Region ap-guangzhou
Access Key AKIXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXf7g
Secret Key gZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZr
Bucket memos-1888888893
Storage Path memos/{year}/{month}/{day}/{filename}

This way, Memos text will be separated from the files.

Webhook

While I have deployed Memos, I still want the previous md data to sync with Memos changes.

Memos also provides webhook configuration, and I wrote a simple webhook server using Flask. When content in Memos is published or modified, it triggers this server, allowing me to operate on the data and append it to the md file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
from flask import Flask, request, jsonify
import logging
logging.basicConfig(level=logging.DEBUG)

app = Flask(__name__)

@app.route('/webhook', methods=['POST'])
def webhook():
    data = request.json
    print("Received data:", data)

    # Here you can add code to process the webhook data

    return jsonify({'status': 'success'}), 200

if __name__ == '__main__':
    app.run(host='0.0.0.0', debug=True, port=5000)

Conclusion

Web-based services are quite enjoyable, as they require no client and can be accessed directly through a browser, allowing for capturing thoughts anytime and anywhere. Additionally, on iOS, Safari can directly send the page to the desktop, making it as easy to open and access as a standard app, providing a great experience.

Moreover, memos also has a very useful RESTful API, allowing it to be extended to other services, such as embedding it into iOS Shortcuts (for scheduling, accounting, journaling, etc.) or integrating it into a blog for more convenience.

The article is finished. If you have any questions, please comment and communicate.

Scan the QR code on WeChat and follow me.

Title:Deploy a self-hosted MEMOS note system
Author:LIPENGZHA
Publish Date:2024/04/13 10:34
Word Count:7k Words
Link:https://en.imzlp.com/posts/30014/
License: CC BY-NC-SA 4.0
Reprinting of the full article is prohibited.
Your donation will encourage me to keep creating!