DaemonForums  

Go Back   DaemonForums > Miscellaneous > General software and network

General software and network General OS-independent software and network questions, X11, MTA, routing, etc.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 4th March 2022
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default Gmail ending less secure apps May 30th affecting .getmail?

I use .getmail to download email from Gmail as an input to Neomutt.

Recently I received email from Google indicating that "getmail" if i understand Google correctly will no longer allow getmail (they term a less secure app) to retrieve mail after May 30th assuming you are currently using it. For those not using it, it is currently turned "OFF"

I found the following link https://www.bytereef.org/howto/oauth2/getmail.html
and was wondering what getmail users out there are doing to comply with Google's
new protocol and if this link information will work for POP3 SSL which i use, or only
effective using imap.

Last edited by frcc; 4th March 2022 at 11:51 AM. Reason: clarification of wording
Reply With Quote
  #2   (View Single Post)  
Old 5th March 2022
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,128
Default

It seems that in the future Google/Gmail will only support OAuth2 for programs/apps to autofetch their mail from their Gmail account. Yahoo mail has already switched.
There is a video at https://oauth.net/ that explains the rationale/"reason why".
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #3   (View Single Post)  
Old 6th March 2022
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Thanks J65nko
Watched the video and of course think 0auth2 is a good idea.

I have followed instructions as annotated here
https://www.bytereef.org/howto/oauth2/getmail.html
(after having diff finding oauth or xauth in the getmail manual)

Attempting to get initial access tokens I get the error message from Google
and Clicking on request for details yields
Code:
"redirect_uri: urn:ietf:wg:wg:oauth:2.0:oob"
consequently not issuing a validation code to enter.

My .getmail gmail.json file is as follows:
Code:
"scope": "https://mail.google.com/",
 "user": "xxx",
 "client_id": "xxx",
 "client_secret": "xxx",
 "token_uri": "https://accounts.google.com/o/oauth2/token",
 "auth_uri": "https://accounts.google.com/o/oauth2/auth",
 "redirect_uri": "urn:ietf:wg:wg:oauth:2.0:oob",
 "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs"}
My getmailrc retriever file section is as follows:
Code:
ca_certs = /etc/ssl/certs/ca-certificates.crt
 type = SimplePOP3SSLRetriever
 server = pop.gmail.com
 port = 995
 username = xxx
 use_xoauth2 = True
 password_command = ("getmail-gmail-xoauth-tokens", "/home/xxx/.getmail/gmail>
 #password =
attempting to start neomutt yields
Code:
  "SimplePOP3SSLRetriever:xxxxxxxxxxxxxxxxxxxxxxxxxxl.com:995:
   getmailrc: operation error (External program error (/home/xxx/.getmail/gmail.json exited with 1))

Last edited by J65nko; 6th March 2022 at 10:24 PM. Reason: Added [code] and [/code] tags .......
Reply With Quote
  #4   (View Single Post)  
Old 7th March 2022
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

update
finally was able to get a token from Google. (Does anyone know where this token goes?)
However now receiving errors concerning username and password from getmail on startup.
The error is getmailrc: operation error (POP error (b'-ERR [AUTH] Username and password not accepted.'))

If any of you have a sample getmailrc file (Retriever Section) to look at i would be interested.

Last edited by frcc; 7th March 2022 at 04:30 PM. Reason: clarify post
Reply With Quote
  #5   (View Single Post)  
Old 8th March 2022
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

No worries, I think i have this figured out. Will post results if anyone interested when i finish.
But, sure dosen't look like there are many getmail neomutt users here, and elsewhere which is why
the doc's for this are sparse. There is a little in the getmail docs but not much.
Reply With Quote
  #6   (View Single Post)  
Old 8th March 2022
scottro's Avatar
scottro scottro is offline
Real Name: Scott Robbins
ISO Quartermaster
 
Join Date: Apr 2008
Location: NYC
Posts: 652
Default

I've been using neomutt and getmail for years. Hadn't heard about this though. If it becomes a pain I may just stop using my gmail accounts. I only use one at the moment and can replace it.
However, I for one would be grateful if you post the results when you get it working smoothly. You can probably write the getmail mailing list but I don't recollect seeing anything about this on there. Seems like right now it's mostly for Gsuite, at least that's the impression I got from the link you posted. I've been wanting to get away from gmail anyway, this may be the incentive I need.
Reply With Quote
  #7   (View Single Post)  
Old 10th March 2022
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Thanks scottro I am still working on it.
Problem is Google isn't the only one doing it now.
It is becoming widespread among mail providers.

It is designed for API developers to share their app which Google considers "less secure" so that their users under test and permanent conditions may use the app (i.e. in this case getmail). For app developers with
a growing market its the cat's meow. In addition, according to getmail it has the same permissions, (0600).
Anyway.....
That being said for single users simply trying to use it for personal use it is a "pain in the xss"
I am well over a thousand keystrokes just trying to understand the process.

I was able to get a token from Google, but have issues from there. According to getmail doc's (which are sparse) it can be done and they have created a script with the latest release to accomplish exactly that. I am trying to get the latest release set-up on one of my machines (linux) to test it.
Also I think there is an issue with upgrading python as well.

That's where i am at presently....I sincerely hope I can solve it because I really like neomutt and have
significant mail stored locally havening been using pop3 for eons.

It appears that usb teathering via ph to desktop has no ill effect on my testing. Also I don't think
.getmail [retriever] section cares in this case weather its pop or imap.

Anyway, since i am certainly not even close to knowing what the xell i am doing it may take a little
time. I will certainly post any updates on this as i dig further.

In the meantime i have had to
"sudo apt-get install thunderbird" as that software as well as many others now have an option
for xauth validation. I have all my mail still available in neomutt but cannot update with getmail.
As a workaround i have added neomutt (mail folders) to local folders in thunderbird but this won't work long term.

Google has warned that if you are using .getmail now without a token i.e. username and password you may still continue to do so until May 30th. if not your blocked from using getmail.
Regards

Last edited by frcc; 10th March 2022 at 02:28 PM.
Reply With Quote
  #8   (View Single Post)  
Old 11th March 2022
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Here is a link that is my starting point, which might help
https://www.bytereef.org/howto/oauth2/getmail.html

In addition i think the latest version that supports this script/option is not available in standard
ubuntu/linux respositories, but of course can be downloaded following instructions via info in the link above. (Evidentially this version is not endorsed or supported by the developer of past versions) further complicating matters.

Anyway still working on it and will continue to report.

Last edited by frcc; 11th March 2022 at 04:36 AM.
Reply With Quote
  #9   (View Single Post)  
Old 11th March 2022
scottro's Avatar
scottro scottro is offline
Real Name: Scott Robbins
ISO Quartermaster
 
Join Date: Apr 2008
Location: NYC
Posts: 652
Default

Thanks for all your work. Presently, gmail is still working normally for me and I'm slowly moving away from it. I'm using ionos.com which is more for web hosting but can do mail as well, which is all I'm using it for.
Reply With Quote
Old 12th March 2022
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default I agree

I understand, I am about sick of Google as well.
Such a dam convoluted system to set-up and nowhere does the docs tell you what to do with their "token"....I now know what they can do with it.............
I have my web servers located on cpanel, so I think i will just forward old mail from gmail to my cpanel website mail system and be done with them. A little work telling people my new email address is about all it takes.
Testing reveals .getmail and of course neomutt works well and easy to configure with cpanel.
Reply With Quote
Old 12th March 2022
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Getmail is very compatible with cpanel which of course is widely accepted in the world.
It took me about 15 min to set up mail on my cpanel server which i have several websites and
goobs of email space. That said if you have a mail carrier that does not force you into xoauth or something similar it would be in your best interests to swap soon. The handwriting is on the wall in regards to (Tm)Google's convoluted complicated goobily gook. I am not in the mood of giving up my use of either getmail nor neomutt as they can be used in a way i want to configure, and, much easier than the nonsense from (Tm)Google.
So, all i have to do now is when new mail comes in, notify sender of my new address. End!
Anyway nuff said!!!

Last edited by frcc; 12th March 2022 at 07:32 PM.
Reply With Quote
Old 12th March 2022
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 1,027
Default

I don't know anything about these technologies, but is this discussion of any help?
Reply With Quote
Old 13th March 2022
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Thanks for the link IdOp
For those wanting to stick with Gmail using getmail it will add additional info for configuration. For me the further i get away from (Tm) Google the better.........
Reply With Quote
Old 14th March 2022
scottro's Avatar
scottro scottro is offline
Real Name: Scott Robbins
ISO Quartermaster
 
Join Date: Apr 2008
Location: NYC
Posts: 652
Default

Heh, I was just about to post about that getmail list discussion. (IdOp had linked to a recent discussion on the getmail list about using oath tokens). As Charles (getmail's creator) wrote they make it harder and harder for the small email providers, probably trying to have them all use gmail.)
Reply With Quote
Old 16th March 2022
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,128
Default

It is not only Gmail. Yahoo mail also uses OAUTH, but seems not even allow this for mail access. From https://eclipse-ee4j.github.io/mail/OAuth2 :
Quote:
The use of OAuth2 with Yahoo services is described here, although it’s not clear that this still works since Yahoo will no longer allow you to create an app with Mail permissions.
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
Old 16th March 2022
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

So far I have had no issues with Cpanel which uses roundcube for its webmail.
I used to host all my websites at my residence with a static i.p. using of course httpd. (OpenBSD).
I enjoyed configuring many programs and of course writing my own firewall. That said My provider was terrible and not many alternatives at the time, therefore I swapped to https://www.ait.com/, web hosting which has the option of utilizing cpanel. I would much prefer hosting my own at my physical location, sadly my concerns regarding local provider up-time etc are still valid. I didn't see the need to use the mail option in cpanel until Google decided to work me over. In addition my mail needs are light as my websites are all static in nature. Anyway I now look forward to eliminating (Tm)Google from my vocabulary as I did 20+ years ago with another pain in the xxx called (Tm) Windows.......................

Last edited by frcc; 16th March 2022 at 05:46 PM. Reason: add wording, clarify
Reply With Quote
Old 21st March 2022
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Update
I don't know how i missed this but here is more info on making .getmail and neomutt play well together with gmail. (Well, I do know, I'm an idiot and over 75 yrs old, so that might explain it!)

Go to your Googlel account settings select "SECURITY"
About halfway down the page you will see App Passwords
Click on arrow on right hand side enter your account password to sign in which will bring you to
Your app Passwords
Simply select app which i choose "mail"
Then select device, I input .getmail
Google will generate a 16 digit code......
copy that code that is highlighted in yellow
Simply paste that code in your password section of getmail and your done
Simply paste that code in your password section when in neomutt and prompted by gmail for your password to have complete access to gmail from getmail and neomutt

I think you need to have 2 step verification turned on in settings as well, i.e. a phone, text verification when accessing gmail accounts. That is easily set-up in accounts as well

Hope this helps those of you using .getmail and neomutt and gmail etc.

Last edited by frcc; 14th April 2022 at 05:45 PM. Reason: format and clarity
Reply With Quote
Old 29th March 2022
jmccue jmccue is offline
Real Name: John McCue
Package Pilot
 
Join Date: Aug 2012
Location: here
Posts: 167
Default

Quote:
Originally Posted by scottro View Post
I've been using neomutt and getmail for years. Hadn't heard about this though. If it becomes a pain I may just stop using my gmail accounts.
I use mutt and that is my plan. I already notified people I may stop using gmail based upon this change. If setting up mutt with gmail becomes a PITA, my gmail account becomes my second 'dump' email (as yahoo is now) and I will move on.

Dump email is what I use when I need to supply an email when purchasing something online.

I use gmail mainly for mailing lists (like OpenBSD misc) and a couple of people still sends me mail there. Most people I care about uses a email I have on my own domain.
__________________
[t]csh(1) - "An elegant shell, for a more... civilized age."
- Paraphrasing Star Wars (tvtropes.org)
Reply With Quote
Old 30th March 2022
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Agree, my last entry has worked with neomutt getmail and gmail. That said i am using that method as a way to give me more time, as you have, to transition to my servers on line or as many users have to another carrier. I can't see moving away from neomutt as i have used it for years and just does what i want with a little tweeking.
Reply With Quote
Old 29th April 2022
scottro's Avatar
scottro scottro is offline
Real Name: Scott Robbins
ISO Quartermaster
 
Join Date: Apr 2008
Location: NYC
Posts: 652
Default

ARGG. Just got the email from gmail about how it will be harder to use after May 30th. I don't know whether it's all Google's fault or all the spammers and phishers who make it a problem. Anyway, my BIG thanks to fcc for explaining it clearly. I'll probably try it over the weekend, in the interim, letting family and such know my ionos email, and telling them to use it.

Last edited by scottro; 29th April 2022 at 12:34 PM.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Please help me automate getmail with cron cssgalactic FreeBSD General 2 9th July 2008 10:13 PM
How secure are apps that using RPC portmapping? aleunix OpenBSD Security 4 9th June 2008 05:53 PM


All times are GMT. The time now is 09:25 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick