I think this is a pretty niche demand and probably another topic for r/DataHoarder but anyway, here I am.

I created this application to basically have a way to store my WhatsApp messages away from the Google/Meta servers. Or at least not depend so much on Google backup.

Whatsapp has a very limited export functionality, which any user can use through the app’s own interface. Once these messages and media have been exported, you can place them in a folder monitored by ChatVault, send them to an email monitored by ChatVault or upload them via the interface. Once ingested by chatvault, it will record the chat media on disk and save the messages in a database in a structured way. These messages can be accessed in a front end similar to a chat application.

It’s still under development, some things need to be improved (mainly the UI), it’s still far from ideal, it’s true, the way Whatsapp allows us to export messages is quite bad, which makes the entire process of exporting and ingesting it into chatvault quite coupled but it can still be useful for those who want to store their messages independently, just like I wanted.

https://github.com/vitormarcal/chatvault

Edit: add an application interface image

The UI still needs some work, but it serves the purpose

  • teslawhytho@alien.topBEnglish
    0·
    10 months ago

    Please don’t take this as criticism, this is a great idea and I fully plan on contributing to the codebase.

    With that said I spent a few hours trying to get it to work. No luck. Docker, no. Docker compose. No.

    I took the code and built / run manually. That worked but then I couldn’t import a chat. I tested with one line with no attachments. From just that one line, here are the problems so far:

    • it doesn’t seem like WhatsApp has a standard way of exporting the text file. Your text file and my text file are different. In the US the format is [datetime] name msg. In your file it’s different and so it breaks the moment it hits the [.
    • unfortunately not accounting for locale. US stupidly uses mm/dd/yy. You have hardcoded the formatter for dd/mm/yy. Maybe you need to have a locale selection in the UI before import. Without that, no US messages are coming in.
    • it doesn’t account for 6/6/23. It’s expecting 06/06/2023. Again formatter and padding can fix that.
    • ui creates an entry in chats table for every attempt regardless of if a message was imported.
    • exceptions in other languages.
    • missing tests for the stuff above

    Again none of that is supposed to be criticism. This is a great idea and I fully intend to help out with it.

    Good job!

  • SnooPets20@alien.topBEnglish
    0·
    10 months ago

    I have a WhatsApp backup zip file. This won’t work, right? Since I think it’s encrypted.

  • siddharthal@alien.topBEnglish
    0·
    10 months ago

    Couldn’t get it to work. Tried running it in docker. Getting this error

    :: Spring Boot :: (v3.1.2) 2023-11-15T17:10:53.882Z INFO 1 — [ main] dev.marcal.chatvault.Boot$Companion : Starting Boot.Companion v0.0.1-SNAPSHOT using Java 17.0.9 with PID 1 (/app/chatvault.jar started by root in /app) 2023-11-15T17:10:53.976Z INFO 1 — [ main] dev.marcal.chatvault.Boot$Companion : No active profile set, falling back to 1 default profile: “default” [Too many errors, abort]

    • NeoJackOfBlades@alien.topOPBEnglish
      0·
      10 months ago

      I just cleared all images and containers to make sure I wasn’t working with any cache and that everything went fine.

      This message is expected because we are not defining any profiles: No active profile set, falling back to 1 default profile: “default”.

      Then I would have to see what error happened after that. The only properties required are the database connection properties.That being said, you can run compose.yml in the project root.It will build an image locally.Or replace the build: ./ line with the image: ghcr.io/vitormarcal/chatvault:latest

      docker-compose -f compose.yml

      I tested these two ways here and they continue to work. Have you added database information?

      • siddharthal@alien.topBEnglish
        0·
        10 months ago

        I tried with this setup only. This was my docker setup.

        chatvault: image: ghcr.io/vitormarcal/chatvault:latest restart: unless-stopped environment: - SPRING_DATASOURCE_URL=jdbc:postgresql://postgres:5432/chatvault - SPRING_DATASOURCE_USERNAME=${POSTGRES_USER} - SPRING_DATASOURCE_PASSWORD=${POSTGRES_PASSWORD} ports: - 8106:8080 volumes: - ‘~/chatvault:/opt/chatvault’ - ‘~/chatvault/config:/config’ depends_on: - postgres

        • NeoJackOfBlades@alien.topOPBEnglish
          0·
          10 months ago

          I’m sorry about that, is your Operating System Unix or Windows? x86 or arm? I tested it on Ubuntu server and fedora and it is correct, I will test it on Windows soon. I’m trying to imagine what it could still be.

  • harriershmarrier@alien.topBEnglish
    0·
    10 months ago

    How does the importing work? I have a whatsapp folder containing logs.zip, Databases, Backups, Media. I put the whole folder into the import folder of chatvault but execute disk import does not seem to do anything

  • katroome@alien.topBEnglish
    0·
    10 months ago

    I worked on a similar project for a long time…
    I don’t really have code that I can share right now, but perhaps I can offer some hints.

    Instead of exporting chats from the app (since the function is too limited) I decrypted the backup database that WhatsApp creates. This was originally possible only on rooted Android phones, but it’s now possible if the user has chosen to encrypt the backup (and the encryption key is known).

    Once the database is decrypted, it is just a matter of extracting messages and media information from it, and rebuild the chat. One of the problems here (and one of the reasons why I could not keep spending time on this project) is that Meta changed the structure of the database multiple times over the last years.

    Here you can see an example of a chat I recreated from the database. The display format is HTML, trying to replicate the WhatsApp original look.

    ​

    ​

    Let me know if you want more info. I will be happy to share!