GMail and POP3 RETR problem - switch to IMAP? - language-agnostic

When I'm accessing GMail inbox using POP3 protocol, it seems that after fetching given email using RETR command, after QUIT-ting and reconnecting, previously RETR-ieved email is not listed anymore when calling LIST.
Then, after going to: GMail settings//Forwarding and POP/IMAP and setting "Enable POP for all mail (even mail that's already been downloaded)", on next login all emails are being LIST-ed again, but if I RETR any of them, it again disappears from LIST after re-logging..
I can then go to GMail settings again and repeat the whole process, but it's a show-stopper for me as I'm writing a script that should work without any manual actions.
Am I missing something, or only IMAP can help here?
(EDIT: RFC http://www.ietf.org/rfc/rfc1939.txt doesn't say a word about RETR command deleting messages)

This is intended behaviour of Gmail. According to this question, "[a]ll messages may be downloaded to another computer once; after downloading mail, it will not download again."
There's also a 'recent mode', in which the last 30 days of mail are fetched, regardless of whether it's been sent to another POP client already.
That said, don't try to fetch all your mail by different computer in a short period of time, as Gmail may block your account for 24 hours.
I strongly suggest using IMAP.

Gmail’s POP3 configuration maybe sometimes confusing. You can find Gmail POP3 behavior here.
Switching to IMAP is very good solution.

This is a common problem, unfortunately, it does not always have the easiest solution. Hopefully, this information will help you and others arrive at the best implementation that suits your needs. Disclaimer: if you have the option or capability to add IMAP to your pop3, it certainly makes things more manageable.
Gmail has its own Pop3 implementation, and with that said, not all of this is relevant to other pop3 providers
Here is the lifecycle of the issue and some information that can help you manage it:
You connect to the pop3 server either in NORMAL mode or RECENT mode. This puts the "session" on the pop server into a "transaction state".
Recent mode is used by prefixing the username on connection with "recent:" + Username. Recent mode will return the last 30 days of email on the server. Note* this will supersede the UIDL command which I will touch on below. I.e. recent mode will return all 30 days worth of email if they have not been removed. Since it always returns the last 30 days, if you have multiple clients, they will all receive the same information in recent mode.
Normal mode is the default. Normal mode will respect the limitations of the commands you choose to use. UIDL will return a chunk of roughly 250 of the oldest emails on the server. If you have 500 emails on the server, and you do not remove any, UIDL will return the id and Unique Identifier for those first 250 emails regardless, so you may not be aware of the new 250. The caveat here is as follows, GMAIL has an option on the web console where you configure pop, to "Enable pop from now on". By selecting that and saving, the timestamp at that moment will be used by the pop server to "refresh" the oldest time. Therefore UIDL will start returning messages back you from that point on up until you reach the 250 mark again (assuming you have not removed them).
It is important to note that the transaction state exists until you issue the QUIT command. Upon issuing that command the server enters the "Update" state, where it will begin issuing the updates you requested, like DELETE commands, or popping them after they have been downloaded. Until QUIT is issued successfully, nothing will be deleted and the server state does not change.
STAT command will show you the number of emails in the pop3 stack, that are on your server.
RETR command will retrieve, or download the email, but it is not marked as downloaded until you successfully end the session
UIDL which many developers use to retrieve the message numbers and unique identifiers is very useful if you maintain the state of the server and pop the email. UIDL will only ever return the oldest 250-ish (I have seen 251-255) emails. If you are constantly polling for new email, this is dangerous if email hasn't been removed. ALSO! if you need to delete email, make sure the GMAIL setting to, Keep a copy in my inbox, is configured in the web console, so that you have access to those emails as a backup.
LIST command would solve your problem in normal mode for getting more than 250 emails back, (note: you still need to maintain an id file locally to cross-check incoming mail in order to know that it is new or old)... HOWEVER: this command also returns mail from the SENT box, which for many, is not a viable solution.
Hints:
If you are managing the inbox quickly and effectively and do not believe 250 to be a limiting factor in your process, UIDL and RETR will work.
If you will not be able to keep your inbox below 250 but also need access to new email, AND you do not expect the inbox to grow to outrageous size and the performance is not concerning, RECENT mode should work.

Related

How to transfer Google Cloud project ownership from MyEmail#mydomain.com to myemail#mydomain.com?

For years, I've been using the email addresses MyEmail#mydomain.com and myemail#mydomain.com as if they are identical. And most of the time this is true. However now the OAuth verification process for the project seems to be failing because Google treats these as two separate identities.
The GCP project owner is MyEmail#mydomain.com. In the OAuth consent screen, I've set mydomain.com as the sole authorized domain for my app. And I use myemail#mydomain.com as my identity in Google Search Console when verifying that I am the owner of mydomain.com.
I got an email from the "The Google Cloud Trust & Safety Team", saying that the owner of the GCP project and the identity of the owner of the authorized domain do not match! The only reason for this seems to be the case of the email names, because everything else appears set up properly.
MY QUESTION: How can I change the GCP project owner from MyEmail#mydomain.com to myemail#mydomain.com?
It seems that I need to change one or the other. I would rather change the GCP owner to myemail#mydomain.com. But I can not get that to happen. I followed the instructions in Grant or Revoke Role.
I go to IAM -> Permissions - Add. I enter the email without the caps & ignore their suggestion to use the one with caps. But in the "Select a role" dropdown, it shows "Owner" as a role "Currently used". I select it anyway and click Save. But IAM -> Permissions never get changed.
I've thought of changing the owner first to someone completely different and then to the lower case email. But that might involve billing emails changing, etc.
EDIT - As a result of trying to add myemail#mydomain.com to the project, I received an email at that address from GCP, asking me to join the project. I accepted the request, but IAM is still only showing MyEmail#mydomain.com as being on the project.
Is this really the case that myemail#mydomain.com and MyEmail#mydomain.com are separate GCP identities? Might there be a different reason for Trust & Safety to think they're not the same?
If I respond to the T&S email, describing my issue, will a real person actually read it, or will the same automated test be run again to check the issue?
Resolution: I responded to the T&S email, explaining what was going on with the upper/lower case letters in my email address.
Today I got a reply: "Request Granted. Your project is now verified for ....". That's great! But I wonder if I will forever be first rejected for the same reason on all new projects that I create. It appears that the final solution is likely finding a way to change my logon email on GCP to one without capital letters.
Since you mention that you are never asked to select a different profile when logging into your account, then it should be the exact same account using the actual same GAIA ID as mentioned by DazWilkin, so there should be no difference within the GCP console between MyEmail#mydomain.com and myemail#mydomain.com.
Google usually recognizes an email address in both forms as the same account, although there are some exceptions across their products (I have had a similar experience with email addresses from Google Groups). I think this is one of those particular exceptions.
I would strongly recommend transferring the project ownership to a totally different account within your domain, then waiting a couple of hours due to Google's "propagation time" across services, and transferring the ownership back to the account using the format myemail#mydomain.com.
Now answering to:
If I respond to the T&S email, describing my issue, will a real person actually read it, or will the same automated test be run again to check the issue?
They are actually a team of people, but they tend to use a lot of canned responses, so I would definitively recommend being very specific with your choice of words when responding to their emails otherwise, you may not get a relevant response. You may also try to explain this to them via email to see if there is an actual problem with the email address or if it is just them or the system being extremely picky when checking the email address.
I think you basically have it covered. But it is important that on new Owner's account, you will need to go to "Billing" in the "hamburger" menu and either link the project to an existing billing account or set up a new Billing account to link the project.
You may also need to delete the old project owner to avoid confusion.

Playwright - "Verify it's you" message only for chromium, while trying to login to Google

I'm writing a Playwright test that starts with a Google Auth0 login. After I fill my test user and password in the UI (google login), in Firefox and Webkit the authentication passes successfully, while, on Chromium, I'm getting the Verify it's you message (with a "send sms" message).
The account does not have 2 steps authentication.
When it happened locally, I opened the browser in headful mode, and after few clicks (which I assume "told" the browser that I'm a real user) the problem disappeared (I can now run my tests in headless mode locally). But, it still happens on CI (GitHub)
I run the test with chromium flags: --disable-dev-shm-usage and --disable-web-security.
I couldn't find any data about it anywhere...
When Google determines that a user is logging in from an unknown device or a new location, they may prompt the user with an additional login challenge.
The login challenge that the user receives depends on the information that associated with the account.
Does the prompt say "Enter a phone number to get a text message" or something else like "This device isn't recognized..."
If the former I believe you can circumvent this extra prompt by having a phone number linked to the Google account in question. If the latter I believe the prompt is once per user per device.
My understanding it is basically Google trying to get a valid phone number for the account (to prevent spam etc).
-- Edit
The only other thing I can think of is that you can temporarily turn off the verify-it's-you challenge, for 10 mins, but only if the account is a member of a Google Workspace or Cloud Identity service. I am not sure this is possible for an unmanaged account - or how useful it would be. The other issue is that for "free services" Google doesn't really offer any kind of support.
Anyhow, you might try "Temporarily turn off login challenges for a user" -
https://support.google.com/a/answer/12077697
There is also so good information on this verify-it's-you challenge here.
https://workspaceupdates.googleblog.com/2018/04/more-secure-sign-in-chrome.html
It has some notes on disabling the challenge per organization via response headers, but again this is for an organization and managed accounts.
If you wish to disable the new screen for your organization, you can
use the X-GoogApps-AllowedDomains HTTP header to identify specific
domains whose users can access Google services. Users in those domains
won’t see this additional screen, as we assume those accounts are
trusted by your users. This header can be set in Chrome via the
AllowedDomainsForApps group policy.

Some users get an error and weird URL when trying to access my Google Script. Debug Tips Welcome

I wrote two small Google Scripts that present simple forms to fill in. Most of my user community has no trouble using them. A small minority of users can never open the forms, instead they get "Sorry, unable to open the file at this time" error page for both forms. I can't find any common thread for why only some users fail. I've tested on multiple browsers on multiple machines, even on android devices, it never fails for me.
A couple of things I've noted:
when it fails for them the URL is re-written. The proper url starts with https://script.google.com/macros/s/... but for broken users when they paste that in they instead get https://script.google.com/macros/u/3/s/... (notice the "u/3" at the end)
There is no execution log created when they try to access the site, so I have no way to debug what's going on.
The app is permissioned so "Anyone" can access it, and it runs as my account
Sorry, I realize this problem description is impossibly vague. Any debug suggestions would be extremely welcome. I'm not a regular Google App Script developer, so I'm kinda stumbling in the dark with this one. Thanks in advance.
/u/3 means that the user have signed-in into multiple Google accounts, the number correspond to the zero-based index of the account in the order that the user followed to sign-in, 0 is for the default account, 1 is to de second account, 2 is for the thirds account and so on.
So, on your test include this use case, a user signed-in into multiple Google accounts.
NOTE: It's known that the HTML Service do not handle as expected this use case.
Related
AuthMode gets confused w/ multiple logged in users
We're sorry, a server error occurred while reading from storage. Error code PERMISSION_DENIED
Why is my script pushing an incorrect URL? [/u/2 inserted into script URL] (possible duplicate)

ejabberd behavior b/w user disconnected vs user unavailable

What is the ejabbered behavior for user who is un-expectedly disconnected from internet
vs
user who explicitly sent an 'unavailable' presence?
Would they both be considered offline (for both single user chat or MUC)?
I want a behavior where if a user is disconnected from internet, offline messages to be sent
If user sent a explicit unavailble presence, I dont want offline messages to be sent.
How can that be accomplished? I can write my hook. But I need to know in which situations, the hook will be called.
When the user gets offline, the default behaviour is the same, no matter which method is used (explicit session close or unvoluntary disconnect). This is per XMPP specification.
If you want to customise the behaviour, it will not be easy as there is no way to know the reason why a user if offline.
What I would do: I would use the last module and support an optional reason for disconnect and store it. When you disconnect unvoluntarily, I would modify code to store reason being something like "timeout". When you disconnect explicitely, I would store another flag. When you get an offline message, you can then check the reason from being offline coming from mod_last storage.

Keeping Email Message from Grouping into Conversation View in Gmail

I'm working on a feature for a client to send them email updates whenever a specific event occurs on their site. When the message shows up in Gmail, the messages get grouped together in conversation view even through they aren't the same conversation. It appears that this is due to the fact that Gmail groups based only on the subject. The client is adamant that we not change the subject line (don't get me started).
Does anyone know how I can disable this by sending a special header in the mail or am I out of luck?
There appears to be no way to prevent this, short of turning off conversation view (have you considered that?).
My guess is that Gmail is actually threading based on its own Thread-Topic header field, which it adds (overwriting any value you pass; it just copies the Subject field) - there's no way of telling, though, unless you can change that field after the fact. Which leads to the suggestion of writing an IMAP application to download the message, edit the headers, and re-upload it again. You'd need to investigate the feasibility of this, though.