API route to export avatar as PNG file is broken

[Alys]

The Data Export - Render a user avatar as a PNG file API route is not working.
https://habitica.com/export/avatar-:uuid.png (for example, https://habitica.com/export/avatar-d904bd62-da08-416b-a816-ba797c9ee265.png ).

It redirects to https://habitica-assets.s3.amazonaws.com/avatars/d904bd62-da08-416b-a816-ba797c9ee265.png which shows an error message

I'm marking this as suggestion-discussion until we've identified what URL it should redirect to.

[librarianmage]

IIRC @lemoness said that there were designs needed about this? The avatar export used to display the level indicator as well, which no longer is on the actual avatar.

[Alys]

Hmmmm. I feel it would be best if it exported the avatar exactly as it's shown on the website but @Tressley what's your intention for this?

[Tressley]

This is actually the first I'm hearing of this feature. I remember some discussion around sharing your Avatar to social media, but this sounds a little different. @Alys -- You're referring to the API route and not an actual site feature, right?

As for how I think we could handle the feature -- This could be as simple as a modal that pops up and lets the user customize their Avatar export with (background, equipment, pet, mount, level) or we just export the Avatar as is with no user customization (we would add level.)

[Alys]

Yes, this is an API route, not a website feature.

It's not related to sharing to social media (at least not directly; you could use the API route to share your avatar image manually).

All this route is meant to do is give you your avatar image as a PNG file. The route can be used directly within third-party tools to display the user's avatar so if the route were changed to produce a modal by default, that would be a breaking change for the API (i.e., something to be avoided unless we gave the third-party developers significant notice that the API's behaviour was changing). We could add an option to the route to allow customisation, as long as the default behaviour was that you'd get a PNG file with no options needing to be provided.

However since this route is currently broken, I suggest we do a simple fix now to bring back the behaviour that used to exist on the old website (i.e., make the feature functional again to give you an avatar image that's identical to the image you see in the website). After that as a lower priority change we could enhance the feature to provide customisation to choose whether to display background, equipment, pet, mount, level, etc.

[carbonsam]

I'll throw my hat into this ring as well - I've noticed that both PNG and HTML avatar exports via the API are broken: {"success":false,"error":"InternalServerError","message":"An unexpected error occurred."}. Looking forward to seeing the fix!

[Majed6]

Same thing here : (

[Aekschen]

confirm

[JacobFrericks]

I like the solution given by @Alys. What else needs to be discussed before we remove the discussion tag and devs can start working on it?

From what I can see, the website takes all the pieces (hair, eyes, armor, pets, etc) and puts them together like a puzzle... but I'd really rather not have to do that. A simple *.png would work nicely.

[Alys]

Since there's been no further discussion, let's settle on the simple fix of bringing back the behaviour from before the new website was released (i.e., the feature should give you nothing but a PNG file that shows your avatar exactly as you see in the website).

How that's done is probably a question for the staff. The feature used to redirect to (for example) https://habitica-assets.s3.amazonaws.com/avatars/d904bd62-da08-416b-a816-ba797c9ee265.png . Is something like that still a viable option? Or should the png be created on the fly and not stored on any webserver (but with the user being able to save it locally of course)?

[paglias]

cc @TheHollidayInn

This is the type of stuff that we should really avoid doing directly on the server and should be moved to a separate worker process

[TheHollidayInn]

Do we send the files to AWS somewhere? Or did we remove that

[paglias]

there's https://github.com/HabitRPG/habitica/blob/develop/website/server/controllers/top-level/dataexport.js#L194 that generates a page with only the avatar but has never been updated and https://github.com/HabitRPG/habitica/blob/develop/website/server/controllers/top-level/dataexport.js#L227 get a screenshot and send it to aws

[T-Zahil]

Any news on this issue ? Currently, I believe there is no other way to retrieve the user's avatar from a third-party tool :/

[paglias]

@T-Zahil I'm sorry but right now we don't have an ETA for this. You can look at how the website builds the avatar and try to replicate that behavior

[Majed6]

*@**paglias <https://github.com/paglias>is it fixable by none-staffers? does require access for some sort of configuration or is it purely backend type of bug ?*

…

On 1/3/2019 1:30 PM, Matteo Pagliazzi wrote: @T-Zahil <https://github.com/T-Zahil> I'm sorry but right now we don't have an ETA for this. You can look at how the website builds the avatar and try to replicate that behavior — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9489 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AG5kn6fS8kkt45-paF3j2ZmH_8K9d6Cmks5u_du9gaJpZM4QfAsD>.

[paglias]

@Majed6 it's fixable by non staffer and it won't require config access but the old method that required to take a screenshot of a webpage from the server isn't very efficient and should be avoided

[Majed6]

@paglias I appreciate your response. How else would we provide a PNG file if the image isn't generated in the backend ? Would you consider using a different library for the screenshot with better performance than Pageres a fix ?

[librarianmage]

I mean, it is possible <https://github.com/tsayen/dom-to-image> to generate pngs on the client, but this does not seem the best place to do that.

…

On Tue, Jan 8, 2019 at 11:13 AM Majed6 ***@***.***> wrote: @paglias <https://github.com/paglias> I appreciate your response. How else would we provide a PNG file if the the image isn't generated in the backend ? Would you consider using a different library for the screenshot with better performance than Pageres a fix ? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9489 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALz_86nGBFPpedHcn2iAj0hsAIniDlu6ks5vBNHUgaJpZM4QfAsD> .

-- Ray A.

[paglias]

Generating PNGs from the client won't probably work as the avatar must be available as as PNH image through the API...

I guess it's more complicated than I initially thought and will require some ops to be moved to a different server, sorry about that @Majed6 but I think this will have to be handled directly by staff

[atlas-comstock]

Seems the api still broken. Not work:
https://habitica.com/export/avatar-29bfa638-cb88-40e8-a0d0-48f455e7b2bd.png

[Alys]

Just for the record:
#12031 causes an informative error message to be shown (linking back to this issue) instead of the "An unexpected error occurred" message.

[evolve2k]

On 1 April 2020 in e39eafd, @SabreCat commented this feature out and added new message referring people to this thread.

{"success":false,"error":"NotFound","message":"This API route is currently not available. See https://github.com/HabitRPG/habitica/issues/9489."}

[evolve2k]

I'm not that familiar with this code but for those that are interested here's the basics of what the code used to do.

Short summary of how the PNG export works

.exportUserAvatarHtml

• When you look up /export/avatar-:uuid.html
• It looks up a User by the provided :uuid and selects the main attributes.
• Renders 'avatar-static' setting the user to the newly looked up details
• Returns html for avatar-static including a title set to the users profile name.

.exportUserAvatarPng then uses the above code to create a PNG

• When you look up /export/avatar-:memberId.png
• It runs the PageRes which converts html to png passing it the results of exportUserAvatarHtml
• It then uploads this png to an AWS bucket, with the url named after the user.
• It then returns the AWS S3 url of the newly uploaded png.
• If a png has already recently been saved in the last 30 minutes, it just returns the url rather than run all of the above.

Adding the above so it's easier to talk about what to do next.

[evolve2k]

Removing the need to use AWS S3

I'm thinking that maybe one issue with this code is that an appropriate AWS S3 account needs to be maintained and the more that this API is used the more files are saved to S3, driving up costs for core maintainers.

What about if we instead Base64 encoded results of the image conversion and returned the Base64 string, which they could then use in an < img > tag to display?

Eg.
< img src="data:image/png;base64,iVBORw0KGgoAAA..." >

I'm thinking this would eliminate the need for an S3 account.

[evolve2k]

Further, if conversion from HTML to Image with Pageres is expensive server side, what other options are there?

Thinking out loud, what if we converted the html into base64 and outputted that?
I'm imagining this might be less intensive.

Finally .exportUserAvatarHtml has been commented out, what's stopping that from being available?
It's not running Pageres or involving AWS, so I can't see the same issues with it being available.

Can we reactivate .exportUserAvatarHtml?

[paglias]

Finally .exportUserAvatarHtml has been commented out, what's stopping that from being available?
It's not running Pageres or involving AWS, so I can't see the same issues with it being available.

Hi @evolve2k the problem with exportUserAvatarHtml is that its only usage was to render the avatar by itself in order for pageres to be able to take a screenshot of the page. Since pageres isn't available anymore it's not very useful to have it, since the avatar is available in the header at Habitica.com.

It also relied on some old render code that is not available anymore, we can add it back if there's a need for it though.

As for the rest of the issues the problem wasn't the cost of AWS s3 but the fact that using Pageres directly in our webserver slowed down all the other operations. We have plans to re-enable it but it'll require a different infrastructure to be setup (a separate worker server to process this type of jobs) and there's no current ETA.

Thinking out loud, what if we converted the html into base64 and outputted that?
I'm imagining this might be less intensive.

Unfortunately that's not possible because HTML converted to base64 can't be rendered as an image

[evolve2k]

Thank you greatly for your detailed response @paglias.

I think as an interim if you could renable exportUserAvatarHtml that would be very useful.

Right now for what I want to build, without the ability to show the users avatar it's really not worth doing, so for the sake of your API ecosystem, having some way to get to the avatar at least gives us a way to keep moving.

With the html output, it provides the foundation for a workaround where Pageres or equivlient can be run by us as the consumers of the API. This then lets us as the rest of the community develop a few simple workaround, like a simple package/code example that runs Pageres on the client side for example.

Unfortunately that's not possible because HTML converted to base64 can't be rendered as an image

What I meant here was convert html to an image and then convert the image to Base 64 encoding, hence allowing the image to be shared as an (encoded) string. Not so relevant if AWS was not the issue.

[paglias]

I think as an interim if you could renable exportUserAvatarHtml that would be very useful.

We're happy to accept a PR that restores the old functionality but uncommenting the code won't be enough as it wasn't already working. Also the avatar rendering code is available here https://github.com/HabitRPG/habitica/blob/develop/website/client/src/components/avatar.vue in case you want to re-use it in a separate project fetching the data with the API. I know that for example the chat extension already does something similar to show avatars https://github.com/HabitRPG/habitica-chat-extension