[CLOSED] Self-Hosted 2.0.0 - Bugs & infos

When testing 2.0.0 here is what i found

Bugs

• New console does not allow to create new orgs (multi-org visible, but the create button is missing)

Others infos

• In 2.0.0 VectorsDB/DocumentsDB are not visible in the UI, it is intended and will be released in a later version once it's more stable. So i would recommend disabling embedding as well to free up some resources while thoses DBs are disabled.
• When using S3, and when in the executor DSN you use the url= param, you should remove the bucket name from the URL. The executor now reuse the one saved in the database path. (e.g. s3://{{ $accessKey }}:{{ $secret }}@localhost/?region={{ $region }}&url={{ $endpoint }} instead of s3://{{ $accessKey }}:{{ $secret }}@localhost/{{ $bucket }}?region={{ $region }}&url={{ $endpoint }})
• When upgrading the instance from 1.9.6, in addition of the migrate command you also need to run the usage-setup command. Else the usage won't work and throw error 503.

Workarounds

Create org from API

# Login
curl -sS -c cookies.txt -X POST 'https://YOUR_HOST/v1/account/sessions/email' \
  -H 'Content-Type: application/json' \
  -H 'X-Appwrite-Project: console' \
  -d '{"email":"YOUR_EMAIL","password":"YOUR_PASSWORD"}'

# Create org
curl -sS -b cookies.txt -X POST 'https://YOUR_HOST/v1/teams' \
  -H 'Content-Type: application/json' \
  -H 'X-Appwrite-Project: console' \
  -d '{"teamId":"unique()","name":"My Organization"}'

Official fixes

(none at the moment)

What and how i tested

I tested a deployment on 2.0.0 with primary maria or psql databases (primary mongo was tested in 1.9 and connection method didn't change). Also tested migration from 1.9.6 to 2.0.0, tested S3 support and finally tested to make a script to migrate primary maria to psql. (the db migration script will be available in this thread, and in my helm chart repo in the #channel thread once it's done)

For each of the above scenario :

• i created a user
• a database, table and row
• created a storage bucket, and a file, with a check of the preview
• created a function (starter node) and tested to execute it with /ping
• created a site and checked if the screenshot worked

-# I will update this message if i find more bugs or others people find some so we have kinda of a mega thread to resolve all issues in self hosted

The script basicly read your maria tables, map the tables to psql (with types changes from a read deployed psql instance for verification), then move the data.

The process to change the primary DB would be :

• Spin up a PostgreSQL alongside your MariaDB
• Setup your PostgreSQL
• Shutdown Appwrite
• Run the migration
• Change the envs to use PostgresSQL as the pirmary DB
• Start appwrite again
• ONLY IF EVERYTHING IS GOOD (you have all your data and and you can continue doing your stuff), you can now remove MariaDB - tho i would still recommend keeping at least the volume for a week just in case

# requirements
pip install pymysql psycopg2-binary

# run
MARIA_HOST=127.0.0.1 MARIA_PORT=13306 MARIA_USER=user MARIA_PASSWORD=password MARIA_DB=appwrite \
PG_HOST=127.0.0.1 PG_PORT=15432 PG_USER=user PG_PASSWORD=password PG_DB=appwrite \
python3 maria-to-postgres.py --drop-schema

# --drop-schema wipes the PG target appwrite schema first, not needed.

Regarding the databases stats empty, only in the database grid view, here is the AI summary of the issue :

What’s going on

• The project Usage → Databases page sums databases.operations.reads/writes for the whole project → you see numbers.
• The databases list cards call the same metrics filtered with resourceId = <databaseId>.

In ClickHouse right now:

So the list query finds nothing → empty sparkline.

Why
Row/document handlers do:

$usage
    ->setResource('database')
    ->setResourceInternalId((string) $database->getSequence())
    ->addMetric(...);  // resourceId left empty

They never call setResourceId($database->getId()). On request shutdown, empty resourceId is filled with the project id (fillMissingResource('project', ...) in general.php). Still broken on main the same way.

Gauges (tables/rows/storage from stats-resources) correctly use the database id — only the operations events used by the list cards are wrong.

Nothing to fix in the chart for this. Fix belongs in Appwrite: set setResourceId($database->getId()) (and ideally setResourcePath(...)) wherever DB ops are recorded.

@Ibaraki Douji , Is there a way we can do perf test between mariadb and psql?

I would say nothing prevent you from adding 1000 rows in a database and checking which is faster.
Basicly do a lot of stuff with the SDK and check which on finish first

Yes, the thing is we are having a product and it's not a small one we have B2B and crossed 300+ customers as of now. The console loading and the database is getting a bit of lag now for me.

if your currently on maria, you can still upgrade, and it will be already faster (2.0.0 have speed improvement)

then if you think it's still too slow or you want to use psql, you can run that benchmark locally on the same machine to test between mariadb and postgresql

Yes, will proceed with same also was thinking with plsql i should be able to use a proper orm to interact right with permisions in place or we need tablesdb?

if the permissions your talking about are the appwrite file permissions, then it's managed in appwrite, not directly the database. if your not talking about that, then i think i didn't fully understood your message.

cc @user @Chirag Aggarwal

looks like a mega thread, i read it a bit. anything specific i should focus on here?

most of them are console only issues (we see cloud specific things, or the stats don't shows everything), so could be fixed kinda at any time, and doesn't break how appwrite work.

But there is others that touch the API, which i will say are the most importants like :

• in the databases grid view /projects//databases, the stats shows empty (with this message that explain a bit more : https://discord.com/channels/564160730845151244/1545738972137914398/1545824880807645285 )
• external redis, user/password logins are not possible

As with the database issue, we can not recover the stats from when it was broken (+ usage > database, only shows one DB with the project ID).
And the redis as it's kinda history issue, and according to AI, is already implemented in some places, but not everywhere

stats might need a look from @user

at least that what i would like to see resolved for the next patch version, and more if you have others fixes to do

@Ibaraki Douji https://github.com/appwrite/appwrite/pull/13523

noice, i'll update the message

Workarounds

Create org from API

(avoid since 2.0.0 migrated everything into one org)

# Login
curl -sS -c cookies.txt -X POST 'https://YOUR_HOST/v1/account/sessions/email' \
  -H 'Content-Type: application/json' \
  -H 'X-Appwrite-Project: console' \
  -d '{"email":"YOUR_EMAIL","password":"YOUR_PASSWORD"}'

# Create org
curl -sS -b cookies.txt -X POST 'https://YOUR_HOST/v1/teams' \
  -H 'Content-Type: application/json' \
  -H 'X-Appwrite-Project: console' \
  -d '{"teamId":"unique()","name":"My Organization"}'

Use redis-proxy

If you really need to use a user-pass you can use a redis proxy that was shown in some github issues

https://hub.docker.com/r/sochimab/redis-proxy

2 more fixes

https://github.com/appwrite/appwrite/pull/13622
https://github.com/utopia-php/monorepo/pull/260

that's incredible, thanks Chirag !!

also small question, should email policies be available in SH ? or is it could only ?

if it should be in SH, then maybe another fix to do in priority as it seems the endpoint don't exists

i will open a new thread for the 2.1.0 issues tomorrow, tho for now i found an issue, and it's that with the new S3 API, the UI doesn't show the S3 tab in the Connect popup (like nothing shows how to connect S3 like the cloud have)

for 2.1.0 see : https://discord.com/channels/564160730845151244/1545738972137914398

[Outdated] Self-Hosted 2.0.0 - Bugs & infos

[CLOSED] Self-Hosted 2.0.0 - Bugs & infos