Gmail ending less secure apps May 30th affecting .getmail?

[frcc]

I use .getmail to download email from Gmail as an input to Neomutt.

Recently I received email from Google indicating that "getmail" if i understand Google correctly will no longer allow getmail (they term a less secure app) to retrieve mail after May 30th assuming you are currently using it. For those not using it, it is currently turned "OFF"

I found the following link https://www.bytereef.org/howto/oauth2/getmail.html
and was wondering what getmail users out there are doing to comply with Google's
new protocol and if this link information will work for POP3 SSL which i use, or only
effective using imap.

[J65nko]

It seems that in the future Google/Gmail will only support OAuth2 for programs/apps to autofetch their mail from their Gmail account. Yahoo mail has already switched.
There is a video at https://oauth.net/ that explains the rationale/"reason why".

[frcc]

Thanks J65nko
Watched the video and of course think 0auth2 is a good idea.

I have followed instructions as annotated here
https://www.bytereef.org/howto/oauth2/getmail.html
(after having diff finding oauth or xauth in the getmail manual)

Attempting to get initial access tokens I get the error message from Google
and Clicking on request for details yields

"redirect_uri: urn:ietf:wg:wg:oauth:2.0:oob"

consequently not issuing a validation code to enter.

My .getmail gmail.json file is as follows:

"scope": "https://mail.google.com/",
 "user": "xxx",
 "client_id": "xxx",
 "client_secret": "xxx",
 "token_uri": "https://accounts.google.com/o/oauth2/token",
 "auth_uri": "https://accounts.google.com/o/oauth2/auth",
 "redirect_uri": "urn:ietf:wg:wg:oauth:2.0:oob",
 "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs"}

My getmailrc retriever file section is as follows:

ca_certs = /etc/ssl/certs/ca-certificates.crt
 type = SimplePOP3SSLRetriever
 server = pop.gmail.com
 port = 995
 username = xxx
 use_xoauth2 = True
 password_command = ("getmail-gmail-xoauth-tokens", "/home/xxx/.getmail/gmail>
 #password =

attempting to start neomutt yields

  "SimplePOP3SSLRetriever:xxxxxxxxxxxxxxxxxxxxxxxxxxl.com:995:
   getmailrc: operation error (External program error (/home/xxx/.getmail/gmail.json exited with 1))

[frcc]

update
finally was able to get a token from Google. (Does anyone know where this token goes?)
However now receiving errors concerning username and password from getmail on startup.
The error is getmailrc: operation error (POP error (b'-ERR [AUTH] Username and password not accepted.'))

If any of you have a sample getmailrc file (Retriever Section) to look at i would be interested.

[frcc]

No worries, I think i have this figured out. Will post results if anyone interested when i finish.
But, sure dosen't look like there are many getmail neomutt users here, and elsewhere which is why
the doc's for this are sparse. There is a little in the getmail docs but not much.

[scottro]

I've been using neomutt and getmail for years. Hadn't heard about this though. If it becomes a pain I may just stop using my gmail accounts. I only use one at the moment and can replace it.
However, I for one would be grateful if you post the results when you get it working smoothly. You can probably write the getmail mailing list but I don't recollect seeing anything about this on there. Seems like right now it's mostly for Gsuite, at least that's the impression I got from the link you posted. I've been wanting to get away from gmail anyway, this may be the incentive I need.

[frcc]

Thanks scottro I am still working on it.
Problem is Google isn't the only one doing it now.
It is becoming widespread among mail providers.

It is designed for API developers to share their app which Google considers "less secure" so that their users under test and permanent conditions may use the app (i.e. in this case getmail). For app developers with
a growing market its the cat's meow. In addition, according to getmail it has the same permissions, (0600).
Anyway.....
That being said for single users simply trying to use it for personal use it is a "pain in the xss"
I am well over a thousand keystrokes just trying to understand the process.

I was able to get a token from Google, but have issues from there. According to getmail doc's (which are sparse) it can be done and they have created a script with the latest release to accomplish exactly that. I am trying to get the latest release set-up on one of my machines (linux) to test it.
Also I think there is an issue with upgrading python as well.

That's where i am at presently....I sincerely hope I can solve it because I really like neomutt and have
significant mail stored locally havening been using pop3 for eons.

It appears that usb teathering via ph to desktop has no ill effect on my testing. Also I don't think
.getmail [retriever] section cares in this case weather its pop or imap.

Anyway, since i am certainly not even close to knowing what the xell i am doing it may take a little
time. I will certainly post any updates on this as i dig further.

In the meantime i have had to
"sudo apt-get install thunderbird" as that software as well as many others now have an option
for xauth validation. I have all my mail still available in neomutt but cannot update with getmail.
As a workaround i have added neomutt (mail folders) to local folders in thunderbird but this won't work long term.

Google has warned that if you are using .getmail now without a token i.e. username and password you may still continue to do so until May 30th. if not your blocked from using getmail.
Regards

[frcc]

Here is a link that is my starting point, which might help
https://www.bytereef.org/howto/oauth2/getmail.html

In addition i think the latest version that supports this script/option is not available in standard
ubuntu/linux respositories, but of course can be downloaded following instructions via info in the link above. (Evidentially this version is not endorsed or supported by the developer of past versions) further complicating matters.

Anyway still working on it and will continue to report.

[scottro]

Thanks for all your work. Presently, gmail is still working normally for me and I'm slowly moving away from it. I'm using ionos.com which is more for web hosting but can do mail as well, which is all I'm using it for.

[frcc]

I understand, I am about sick of Google as well.
Such a dam convoluted system to set-up and nowhere does the docs tell you what to do with their "token"....I now know what they can do with it.............
I have my web servers located on cpanel, so I think i will just forward old mail from gmail to my cpanel website mail system and be done with them. A little work telling people my new email address is about all it takes.
Testing reveals .getmail and of course neomutt works well and easy to configure with cpanel.

[frcc]

Getmail is very compatible with cpanel which of course is widely accepted in the world.
It took me about 15 min to set up mail on my cpanel server which i have several websites and
goobs of email space. That said if you have a mail carrier that does not force you into xoauth or something similar it would be in your best interests to swap soon. The handwriting is on the wall in regards to (Tm)Google's convoluted complicated goobily gook. I am not in the mood of giving up my use of either getmail nor neomutt as they can be used in a way i want to configure, and, much easier than the nonsense from (Tm)Google.
So, all i have to do now is when new mail comes in, notify sender of my new address. End!
Anyway nuff said!!!

[IdOp]

I don't know anything about these technologies, but is this discussion of any help?

[frcc]

Thanks for the link IdOp
For those wanting to stick with Gmail using getmail it will add additional info for configuration. For me the further i get away from (Tm) Google the better.........

[scottro]

Heh, I was just about to post about that getmail list discussion. (IdOp had linked to a recent discussion on the getmail list about using oath tokens). As Charles (getmail's creator) wrote they make it harder and harder for the small email providers, probably trying to have them all use gmail.)

[J65nko]

It is not only Gmail. Yahoo mail also uses OAUTH, but seems not even allow this for mail access. From https://eclipse-ee4j.github.io/mail/OAuth2 :

The use of OAuth2 with Yahoo services is described here, although it’s not clear that this still works since Yahoo will no longer allow you to create an app with Mail permissions.

[frcc]

So far I have had no issues with Cpanel which uses roundcube for its webmail.
I used to host all my websites at my residence with a static i.p. using of course httpd. (OpenBSD).
I enjoyed configuring many programs and of course writing my own firewall. That said My provider was terrible and not many alternatives at the time, therefore I swapped to https://www.ait.com/, web hosting which has the option of utilizing cpanel. I would much prefer hosting my own at my physical location, sadly my concerns regarding local provider up-time etc are still valid. I didn't see the need to use the mail option in cpanel until Google decided to work me over. In addition my mail needs are light as my websites are all static in nature. Anyway I now look forward to eliminating (Tm)Google from my vocabulary as I did 20+ years ago with another pain in the xxx called (Tm) Windows.......................

[frcc]

Update
I don't know how i missed this but here is more info on making .getmail and neomutt play well together with gmail. (Well, I do know, I'm an idiot and over 75 yrs old, so that might explain it!)

Go to your Googlel account settings select "SECURITY"
About halfway down the page you will see App Passwords
Click on arrow on right hand side enter your account password to sign in which will bring you to
Your app Passwords
Simply select app which i choose "mail"
Then select device, I input .getmail
Google will generate a 16 digit code......
copy that code that is highlighted in yellow
Simply paste that code in your password section of getmail and your done
Simply paste that code in your password section when in neomutt and prompted by gmail for your password to have complete access to gmail from getmail and neomutt

I think you need to have 2 step verification turned on in settings as well, i.e. a phone, text verification when accessing gmail accounts. That is easily set-up in accounts as well

Hope this helps those of you using .getmail and neomutt and gmail etc.

[jmccue]

Originally Posted by scottro

I've been using neomutt and getmail for years. Hadn't heard about this though. If it becomes a pain I may just stop using my gmail accounts.

I use mutt and that is my plan. I already notified people I may stop using gmail based upon this change. If setting up mutt with gmail becomes a PITA, my gmail account becomes my second 'dump' email (as yahoo is now) and I will move on.

Dump email is what I use when I need to supply an email when purchasing something online.

I use gmail mainly for mailing lists (like OpenBSD misc) and a couple of people still sends me mail there. Most people I care about uses a email I have on my own domain.

[frcc]

Agree, my last entry has worked with neomutt getmail and gmail. That said i am using that method as a way to give me more time, as you have, to transition to my servers on line or as many users have to another carrier. I can't see moving away from neomutt as i have used it for years and just does what i want with a little tweeking.

[scottro]

ARGG. Just got the email from gmail about how it will be harder to use after May 30th. I don't know whether it's all Google's fault or all the spammers and phishers who make it a problem. Anyway, my BIG thanks to fcc for explaining it clearly. I'll probably try it over the weekend, in the interim, letting family and such know my ionos email, and telling them to use it.