MMS table to XML export

[rkr8]

Hey there,

I'm currently trying to figure out how i can convert all the text messages from a Signal backup into a xml file (which should be imported in the default SMS app later). In https://github.com/bepaald/signalbackup-tools#export-xml it says that currently only the SMS table is supported, which excludes sent group messages and media. However, in the code i found:

signalbackup-tools/main.cc

Line 261 in f3999ee

if (!sb->exportXml(arg.exportxml(), arg.overwrite(), false /*include mms*/, SignalBackup::DROPATTACHMENTDATA))

Enabling this flag and running --exportxml gave me the error message ERROR Failed to retrieve required field 'address' (mms database, type = 10485783) and a corrupted XML file. Thus I was wondering if there is any possibility to get this feature working?

Thanks in advance!

[bepaald]

Yes, that would be expected, that functionality is not there (yet?).

The 'address' ( = phone number) field is a required field for the sms xml spec, but messages sent to a group, do not have an address as they are not sent to any single phone number. And in sms, there is no such thing as a 'group'.

I could just duplicate all outgoing group messages as if having been sent to each group member individually. Would you be OK with that? It seems a bit messy to me. Although, with the current behavior (which was a re-implementation of the official xml export) this is already the case with incoming messages: there is no indication that an incoming message from any person was sent to you personally or to you as a member of some group.

Alternatively, I suppose I could just assign some fake phone number (maybe an invalid one to guarantee it will never clash, though invalid phone numbers might be rejected by an importing application), and have all group messages use that number as 'address'. In that case, the group messages aren't mixed with individual messages, but I don't think there will be any way to distinguish between the authors of incoming messages.

Let me know what you think. It might take a little while to implement. Some bugs will also probably come up, so testing will be required.

As for media messages, I think I actually implemented including those, but I could easily just leave them out and just include the message body in the export.

Note also there are many other features that may not be supported by sms (multiple attachments, quotes, emoji reactions), so no xml export will ever be complete.

[rkr8]

Hi,

thanks for your quick response and effort! I think the option to just duplicate all outgoing messages (as if they have been sent to each group member individually) would be very convenient. For me personally, it would be more important to distinguish between the author of the group messages than to distinguish between personal and group message. But others might have different priorities. Maybe having a command-line flag for choosing one of the options would be a good compromise?

If you choose to implement any of that, I could test the feature with my signal DB on Windows and WSL :)

Slightly off-topic, but the whole signal to XML thing is mostly a workaround for me to somehow get the signal text messages from Android to iOS :D I know it's quite cumbersome, but i don't see any other option than:

• exporting signal messages to XML
• importing messages in pre-installed SMS app
• using "Move to iOS" app to import the messages into iMessage on iOS

[bepaald]

Hi! Well, scratch most of what I said in my previous message. Seems like MMS does support actual groups, so maybe I can get a much more accurate XML file out of the backup. I need to do some research though, the only specification I have seems somewhat incomplete or under defined (at least as far as I understand it). (ref: https://synctech.com.au/sms-backup-restore/fields-in-xml-backup-files/ and https://synctech.com.au/wp-content/uploads/2018/01/sms.xsd_.txt). This will probably take a while to implement properly. Is there any particular program you are using to import the XML file?

Slightly off-topic, but the whole signal to XML thing is mostly a workaround for me to somehow get the signal text messages from Android to iOS

Right. That seems like a good reason to want this option. I think I've had messages from other people doing this same procedure (losing some messages due to the limitations obviously).

EDIT
I thought I'd just send myself some group mms and multi-attachment mms to see what they look like in an exported xml file. Unfortunately, as it turns out, the last provider to support mms in my country stopped supporting it in 2019. Never noticed that 😀. It'll make research a bit more time consuming though. Luckily, I did have some normal (single attachment, single receiver) mms messages from years ago already on my phone, so I can at least check those out.

[rkr8]

Hi! Sorry for my very late reply! Thank you so much for the research! So far I've only tested this app https://play.google.com/store/apps/details?id=com.riteshsahu.SMSBackupRestore&hl=en&gl=US which worked very well, but I'm not sure if it supports group MMS messages. Probably this feature is not really a priority for most people, so whenever you find time I could test it on Windows + WSL :)

[jbaker6953]

This functionality is available from https://github.com/alexlance/signal-message-exporter

[bepaald]

I finally got around to testing the current state of the exportxml feature. I don't know when exactly, but it seems I have implemented everything already? In my testing, all (non-status) messages are exported, including all media and group messages.

A few things I noticed while testing.

1. The phone on which an XML backup is restored requires a working SIM card. At least, on mine it does, it may depend on OS type. Without a SIM card, SMS Backup & Restore will fail restoring when the first outgoing message is encountered.
2. While the XML format allows for a contact_name field, which is filled with a group's title, this seems to be ignored by messaging apps. Messages are grouped by the list of recipients (= phone numbers). If multiple groups have the exact same group members-list, the messages from these groups are merged (regardless of differing contact_name values). Again, this may depend on OS or messaging app used (I tested with Google messages).
3. I ran the --exportxml on backups from three different phones. On one, the most recent one, most Signal contacts did not have a phone number associated with them. This was not a problem in the past, as Signal required a (shared) phone number anyway. These days, phone number sharing is optional in Signal (and off by default I believe). Since the phone number is a required field of the SMS-XML format, messages without one are skipped. I solved this by manually setting phone numbers for all contacts in the backup (--runsqlquery "UPDATE recipient SET e164 = '+1234456789' WHERE _id = X", this command can be repeated for each recipient, and combined with the --exportxml option in a single run).

So, as far as I can tell, everything mentioned in this issue is working, and probably has been for quite some time. I will need to update the README to mention the above points in the "export to XML" section, but other than that I think it's time to close this issue.

If anyone still cares about this, and encounters problems, just let me know and this issue can be reopened (or feel free to open a new one).

Thanks!