validation of downloaded snapshots

[scholzmichael]

It often happens to me that the downloaded snapshots are not valid or corrupted, which has nothing to do with the tool I guess.

It would be great to actually check the downloaded files automatically and if corrupted get a fresh download. Don't know if this is possible but would be great.
What am I using the tool for: Downloading snapshots of my 4 devices every 5 minutes, copy the latest ones on my NAS so I can fetch the with my light manager and make them visible on the Cloud WebIF. Automatically delete all old jpeg except the latest one by using shellscripts and crontab.
I used iobroker before with the ring adapter, but every couple of days it stopped working, and I had to reboot the machine. All running on a raspi 4 with bullseye image.

By the way: fantastic work you delivered here!!!!!!!!!!!!!!!

[KoenZomers]

Haven't encountered this myself yet @scholzmichael, but I have looked into adding such a thing and am testing a code change now that would add a feature to validate the downloaded image.

[KoenZomers]

Version 2.0.0.0 has a new -validateimage option. Can you give it a try and let me know if this works for you?

[scholzmichael]

Hi @KoenZomers , thanx a lot, sure will try. Do I just need to exchange the files, or do I need to run it to get a new token?

[KoenZomers]

You should be able to just replace the executable. In rare scenarios I've seen it go haywire with the settings file. If that happens, just delete the settings file, run it again, go through the authentication process with multi factor authentication and you're all set for future runs.

[scholzmichael]

Just started again with version 2.0.0.0 was forced to do 2 factor again, but ran without a problem, now check this:
Validating image... Failed: image content not recognized, retrying (1/3)
OK
Validating image... OK
OK
Validating image... OK
Done
seems to work like a charme, thanx a lot....great work!!!!

[KoenZomers]

Weird that it never happens to me. I also need to look at why it states to validate the image twice. Is this the exact output it gave you?

[scholzmichael]

Yes, thats exactly what I got, copied from console when running my get_snapshots.sh:

#!/bin/bash
cd /opt/iobroker/iobroker-data/files/ring.0
./RingSnapshotDownload -username user -password PW -deviceid XXXXXXXX -forceupdate -validateimage -out cocoa_XXXXXXXX
./RingSnapshotDownload -username user -password PW -deviceid XXXXXXXX -forceupdate -validateimage -out cocoa_XXXXXXXX
./RingSnapshotDownload -username user -password PW -deviceid XXXXXXXX -forceupdate -validateimage -out doorbell_XXXXXXXX
./RingSnapshotDownload -username user -password PW -deviceid XXXXXXXX -forceupdate -validateimage -out doorbell_XXXXXXXX
exit
could be that one of the cameras is taking a video at the time of the corrupted download, surely has nothing to do with your code, rather with the api, since the iobroker ring adapter had the same problem spradicly...

[scholzmichael]

by the way, running the .sh every 5 minutes via crontab, if you have longer times in between you migt not recognise the issue...

[scholzmichael]

New problem, it seems that with every run a 2f authentikation is requested, getting new security codes from ring via sms every time...
Using settings from /opt/iobroker/iobroker-data/files/ring.0/Settings.json
Connecting to Ring services

Authenticating using provided username and password
Two factor authentication enabled on this account, please enter the Ring token from the e-mail, text message or authenticator app:

just happened again, need to stop I guess before ring throws me out...

went back to 1.3

[KoenZomers]

Weird. Did some testing here both on Windows as well as on Linux and I cannot reproduce. Also didn't change anything in that area in the last releases. Here it works consistently well. Even if I let it loop with 5 seconds pauses in between, no problems, no corruptions, no MFA tokens requested. Could it be that the user you run the script with doesn't have write permissions to the directory or the Settings.json file? After a run where you authenticate with MFA, do you see that Settings.json gets updated and shows a base64 encoded value for RingRefreshToken which starts with ey? If that's not the case, the tool cannot write to the file. Check your chmod permissions. If it does update the Settings.json file, on a next run, the tool should report Authenticating using refresh token from previous session and take that refresh token to authenticate. You can also examine that refresh token using jwt.ms. If you paste it in there, it should show you that its a Ring refresh token.

[scholzmichael]

well, right now I can't login to ring anymore:   Unhandled exception. KoenZomers.Ring.Api.Exceptions.AuthenticationFailedException: Authentication of the session failed     at KoenZomers.Ring.Api.HttpUtility.FormPost(Uri url, Dictionary`2 formFields, NameValueCollection headerFields)     at KoenZomers.Ring.Api.Session.Authenticate(String operatingSystem, String hardwareId, String appBrand, String deviceModel, String deviceName, String resolution, String appVersion, Nullable`1 appInstallationDate, String manufacturer, String deviceType, String architecture, String language, String twoFactorAuthCode)   I guess ring finally did throw me out, will login to the official app and wait a while to see if we can get it to work again :(   Confirmed, my account is locked... By the way: it is def. not a problem with rights since crontab runs as root    

[scholzmichael]

OK, getting in again, testing your version 2.1.0.0 manually so that I don't get thrown out again :)
I will comment here if the tool does another 2FA again, now I found the following:

If the jpg is validated as corrupt it downloads it twice according to log, but does not save it to the specified folder:

Using settings from /opt/iobroker/iobroker-data/files/ring.0/Settings.json
Connecting to Ring services
Authenticating using refresh token from previous session
Requesting Ring device to capture a new snapshot
Downloading snapshot from Ring device with ID XXXXXXXX... OK
Validating image... Failed: image content not recognized, retrying (1/3)
Downloading snapshot from Ring device with ID XXXXXXXX... OK
Validating image... OK
Downloading snapshot from Ring device with ID XXXXXXXX... OK
Validating image... OK
Done

what it shoud do at the end is:

Requesting Ring device to capture a new snapshot
Downloading snapshot from Ring device with ID XXXXXXXX... OK
Validating image... OK
Saving image to cocoa_XXXXXXXX/XXXXXXXX - 2023-09-06 08-01-03.jpg... OK
Done

according the .sh I created:

nload -username [email protected] -password password -deviceid XXXXXXXX -forceupdate -validateimage -out cocoa_XXXXXXXX

So it saves it only to the specified folder if the first validation was ok...

[scholzmichael]

so, I could now reproduce the 2FA problem, seems to start after 1-2 hours running with no problem. When requesting a simple "-list" manually I get this:

Using settings from /opt/iobroker/iobroker-data/files/ring.0/Settings.json
Connecting to Ring services
Authenticating using provided username and password
Two factor authentication enabled on this account, please enter the Ring token from the e-mail, text message or authenticator app:
XXXXXX
Retrieving all devices... 4 found

So something def. changed from 1.3.0.0 to 2.0.0.0 (2.1.0.0), settings.json suddenly shows "RingRefreshToken":null"

Is perhaps the case that a new token is needed, but not pulled? Could also be that it only uses user and pw, not the token when moving on...

Seems to happen after time, shortly after doing the above I did a "-list" again and received the following output:

Using settings from /opt/iobroker/iobroker-data/files/ring.0/Settings.json
Connecting to Ring services
Authenticating using refresh token from previous session
Retrieving all devices... 4 found

very strange, I'll wait now for 2 hours and try again....

07.09.2023

Ok, comming closer, started automatic download at 8:50h via crontab, 8 refresh Jpgs were downloaded from each ring device, and everything worked till 9:35h, then I received SMS from Ring with 2FA code, checked settings.json, entry here suddenly shows: "RingRefreshToken":null", every other data such as username and password are still in there.

So somehow without me touching, the token was deleted from settings.json, which ended up in requesting new 2FA code.

Stopped again before account gets locked again.

I guess something must be different from V1.3.0.0, please check for deleting token in your code...

By the way, the settings.json showed last modification and last permission change also at 9:35, but since I did set it to 777 before start I could not find a change in permissions

Went back to 1.3.0.0 running fine and stable now for approx. 7 hours