Ultimately my objective is to prevent email tracking via auto-loaded images with unique links. I'm aware that Google uses a proxy to load the images so at least they won't reveal my IP address, but there are certain individuals and organizations that annoyingly embed trackers into their emails -- and they actively check whether or not I read their emails.
In the Gmail app specifically, I can turn off auto-loading of images, but I'd like to have similar protection for other apps that don't have this setting. For example, apparently there isn't a way to do this with Google Inbox.
My current thought is to write a back-end script that can run on new mail (received either from an event or frequent polling) to turn embedded HTML images into hyperlinks links to those images -- only used if I really need the image. That way, no matter what app I use to open the email, I'm in control of how/when I'm tracked. Editing emails is something that I've done with the MS Exchange Server APIs, and I'm looking for a way to do this with Gmail -- by whatever means available.
I found a couple of threads from 2010 on how to modify the subject line using Google Apps Script and Gmail itself. At the time, you couldn't do that, but it seems possible that these have been updated since then or that there are solutions using the Gmail API or IMAP.
tl;dr
For my Gmail account, how can I programmatically modify (and save changes to) received emails?
Possible solutions:
Google Apps scripts
Gmail API
IMAP
Other?
I think the Gmail API would suit your needs perfectly.
Let's say I'm polling my Inbox every minute for new messages, with the Users.messages.list()-request. I'm careful to use the after-parameter in my query with the value of the last time I checked my Inbox, as seconds since the epoch. I only ask for Ids of the potential new messages. You could also subscribe to push events to mitigate the risk of your user pressing the message before you poll and alter the messages, as mentioned by #Max in the comments. Probably not an issue if the script is just for you.
q = after:<TIME_IN_SECONDS_SINCE_EPOCH_OF_LAST_POLL>
fields = messages/id
GET https://www.googleapis.com/gmail/v1/users/me/messages?fields=messages%2Fid&q=after%3A1437677475478&access_token={YOUR_API_KEY}
Response:
{
"messages": [
{
"id": "14ebc16800d1fdc0"
}, ...
]
}
Ha! I have a new message. I get it raw, decode its URL safe base64-encoded content, and have a look.
format = raw
fields = raw
GET https://www.googleapis.com/gmail/v1/users/me/messages/14eb68cb028163ba?fields=raw&format=raw&access_token={YOUR_API_KEY}
Response:
{
"raw": "RGVsaXZlcmVk..."
}
Let's do the aforementioned base64-decoding. Replace all the "-" with "+", and "_" with "/" to transform it from URL safe base64 data to regular base64-encoded data.
atob("RGVsaXZlcmVk...".replace(/\-/g, '+').replace(/\_/g, '/'));
Result:
<html lang="en">
<head>
<title>
Computerphile just uploaded a video
</title>
.
.
.
<img class="open_tracking_img" src="http://www.youtube.com/attribution_link?a=vi-KC3YA0Qc&u=/gen_204%3Fa%3Dem-uploademail" width="1" height="1">
.
.
.
</html>
Contains a lot of img-tags, for sure.
I just extract the img-tags, get the URLs, and remove all the img-tags in the mail with my favourite XML Parser.
After the tags are removed, I just insert the URLs in the mail where I see fit, and encode it back to the URL safe base64-encoded data it was retrieved in.
btoa("<html lang="en">...".replace(/\+/g, '-').replace(/\//g, '_'));
Finally, I delete the original mail and insert the modified one.
DELETE https://www.googleapis.com/gmail/v1/users/me/messages/14eb68cb028163ba?access_token={YOUR_API_KEY}
POST https://www.googleapis.com/gmail/v1/users/me/messages?access_token={YOUR_API_KEY}
{
"raw": "RGVsaXZlcmVkLVRvO..."
}
My new, modified mail is now in the inbox instead!
Related
I am a total newb to API and json so this might be basic. But couldn't find a solution by googling.
I want to change e-shop order status via API by clicking a hyperlink in an e-mail.
I activated an API and managed to change the order status by Postman by following command:
PUT {url}/api/v2/orders HTTP/1.1
Content-Type: application/json
Authorization: Basic {abcdefgh}
{
"orders": [
{
"order_number": "00001",
"status_id": "16",
}
]
}
Is there a way how to run this command by simply clicking a hyperlink?
And should I be concerned about security since the authorisation is hardcoded there?
CHeerS!
Email clients for safety reasons do not support the execution of scripts or anything else other than a GET request. As this would require the use of javascript/jquery to build up a payload and call the API with said payload.
You will need to take the client to a secure page to sign in and manage their order.
The hyperlink can perhaps take them to a sign-in page or register page.
Token authorization might work with email being the verification taking the user to a page to see their orders. But again, you won't be sure an authorized person opens the email.
Regarding hard coding any type of authorization, that is a big no.
Since clicking a hyperlink in an email is the same as typing out the address in the browser bar, you can't make POST requests through it. One way of doing what you want is to generate a onetime-use token, and simply put it in the url. When the user clicks the kyperlink, the GET request to the server will contain the token, which can be used for validation.
Is there a way how to run this command by simply clicking a hyperlink?
Not in general, no. Clicking on a link in an e-mail issues a GET request, which can't contain a body. That is: your "orders" JSON won't be included. It also won't know to include the Authorization header.
As far as I know, no common e-mail clients allow you to issue PUT or POST requests.
So: could you encode the request in the URL, and use a GET request instead? Absolutely you could. Don't do this.
There are several reasons for this. The most important is the one you mention:
And should I be concerned about security since the authorisation is hardcoded there?
Hardcoding authorization is a bad idea in general, but particularly in an email: (1) you can't guarantee that an email is encrypted, which exposes the credentials to anyone who can capture the message; (2) if you forward the email to me, I now have your credentials.
Moreover, if you include the authorization in the URL, that's now in the user's browser history, and if they share the link with anyone ("hey, look at this deal on paperclips!"...), well: same as above.
Working on a project for my employer. They are searching for a solution to connect JSON to a email template.
For now, I look for a solution where the system who has the JSON, loads the JSON script on a server. Then, we connect that server with a marketing automation system, for example Marketo. We like to create an email template which loads the JSON.
The developer we found, says it is not possible and it will not work in gmail, outlook or any other client. For what I know, it is possible to create a email template with JSON right? Just to be sure, because when it is possible, our problem is not the system, but the developer.
Like to hear your thoughts on this subject. Suggestions are welcome.
So, this can be done, but you're going to need to do some custom development.
Marketo's API includes a request campaign endpoint for triggering Smart Campaigns, including those which send emails. As part of the request campaign payload, you can send {{my.token}} values for the campaign.
If you can set up some middleware between your JSON output that renders the HTML for your email, you can push that rendered HTML as a token in the campaign request, allowing for on-demand email creation based on your JSON content.
There are two caveats with this approach:
You will have to dynamically send the email using Request Campaign. That means if you need to send unique content to multiple folks, you'll be using one API call per email.
Due to the nature of Request Campaign triggers, you will not be able to batch and send using Marketo's controls; however, this should be straightforward to set up on the API side.
You can build an email using dynamic data, e.g. from an API that sends JSON.
However, you cannot send an email that calls dynamic data. The final email that gets sent must be static. That may be what your developer is getting at. Email software will strip out scripts.
I am struggling to understand how our local gov website functions. Specifically the URL https://self.maidstone.gov.uk/service/check_your_bin_day
Beginning to enter a postcode (not mine) such as 'ME15 7HQ' and without submitting the form, a list of addresses is available.
Selecting an address and then selecting 'black bin and food', the website will return a result of 'Your collection date will be FRI 30/07/2021'. Again without submitting the form.
Running OWASP's ZAP, I can see that data is being submitted and returned as JSON. The data begins to be submitted as soon as the postcode is being entered.
I'm obviously an amateur but have never seen a site like this. The purposes of the request to Stackoverflow is that I would like (either via curl or a HTTP request) to be able to submit a postcode and address, and then receive the bin days as relevant.
Is this possible? If so any pointers please?
This website uses JavaScript and HTTP requests to dynamically load data.
For viewing those requests, open devtools > Networktab, and click on XHR button:
Afterwards, try to write postcode into the form. For me, it made request to runLookup url. When you click Preview tab, you can see the JSON you was mentioning. In this case, you can find the Street addresses (which are then added to next input on website) is on integration > transformed > rows_data.
The devtools have one amazing feature, that will convert the HTTP request to cURL command, you can do it by right-clicking on the request, Copy > Copy as cURL command:
By this way, you can make the whole process, although, from seeing the whole complexity of their API, I don't think it will be that easy, due you probably would need to preserve cookies, and generally follow some steps of requests to get the result (but this doesn't need to be true, it's just my observation).
I'm building a Google Script Web App that uses the user's Gmail messages
(I'm using the built-in gmail api), when the user chooses a message, I want to show him the message's content, to asure that's the right message.
I'm afraid the mail might contain xss, but I don't want to loose the styling of the message, it's ugly, so I can't write
var output=HtmlService.createHtmlOutput("");
output.appendUntrusted(message.getPlainBody());
my current Code is:
function getMessageContent(message)
{
var output=HtmlService.createHtmlOutput("");
output.append(message.getBody());
return output;
}
does a mail really might include xss?
If the answer is yes, how can I prevent it?
thanks Noam.
So, I've gotten to the point in my app where I can retrieve a list of spreadsheet documents from a user's Google Drive account by using the Google Spreadsheet API. I populate the file list in a ui control that users can click on to then retrieve the list of its worksheets. It's working as expected in some cases, but in others it is not. In my request, I use the url that comes back from the file list, and even so, the API responds with:
Sorry, the file you have requested does not exist.
Make sure that you have the correct URL and that the owner of the file hasn't deleted it.
Well, surely the file wasn't deleted. I got it back in the response to my request for the files list. I also can get to the file via a normal web browser. Also, the URL is correct because that's the one the API responded with. My code does not manipulate the url that comes back in that initial files response. In fact, here is the URL that is used to grab the worksheets:
https://spreadsheets.google.com/feeds/worksheets/{long key here}/private/full
So, my question is why does my request for some worksheets come back with a response with the actual list of worksheets, but on others (which I have access to, and I know exists) I get the faulty response.
Thanks,
Arie
My app is using OAuth 2.0 and I ran into the same error with new Google Sheets. What fixed that was making a change in scope param sent during OAuth's authorize call and then reauthorising (reinitiating OAuth flow and obtaining new tokens).
Until now scope in my app was just:
https://spreadsheets.google.com/feeds
Updated scope and solution to the issue in my case:
https://spreadsheets.google.com/feeds https://docs.google.com/feeds
I'm running into this in my own stuff. At least for what I'm running into, it seems to be an issue with New Sheets. I'm sorry to not have more of a solution (I'm still trying to find out what to fix on my end) but this may help you narrow down the issue.