Freebox Server (Ultra V9/ Pop V8/ Delta V7 / Revolution V6 / Mini 4K)

  • Status Nouveau
  • Percent Complete
    0%
  • Task Type Évolution
  • Category Téléchargements
  • Assigned To No-one
  • Operating System Freebox V9 (Ultra)
  • Severity Low
  • Priority Very Low
  • Reported Version 4.8.15
  • Due in Version Undecided
  • Due Date Undecided
  • Votes
  • Private

FS#39771 - Gestion de Google Takeout

Bonjour

Google Takeout ( https://takeout.google.com/settings/takeout/custom/photos?pli=1 ) permet notamment d'exporter via des liens de dl le contenu Google Photos.
Ces liens ne sont pas reconnus par la box.
Ca permettrait une fonction de sauvegarde sur le DD de la box de son compte google (photos ou plus large)
Merci !

Admin
Thibaut Freebox commented on 14.10.2024 12:10

Bonjour

Vous parlez du lien généré après les clics (type de ZIP, taille, etc) et qui est envoyé par mail ?

Ou vous demandez à ce que tout le processus soit intégré à Freebox OS/Freebox Files ?

Cdt

Mirx74 commented on 17.10.2024 05:51

Bonjour, uniquement des liens, pas nécessaire pour le process entier je pense.

nbanba commented on 02.11.2024 10:21

Bonjour

Alors désolé de vous contredir, j'ai essayé de faire fonctionner dans l'état en décortiquant le process et la conclusion c'est qu'il faut 100% du processus une fois les liens Takeout.
Je m'explique, le lien ne pointe pas sur le/les fichiers à télécharger, mais il pointe sur une chaine de fichiers javascripts (le premier fait 850 KO, presque 1 mega !!) qui en fonction de la session authentifiée fini par donner accès au fichier

Chez moi le cURL qui download en directe à la tête suivante (désolé pour les XXXX et les xxxxxxxxxxxxxxxxxx, c'est un forum public): 

 
curl 'https://takeout-download.usercontent.google.com/download/takeout-20241101T091938Z-001.tgz?j=380XXXXX-6XX5-4XX2-9XXd-dXXXXXXXXXbf&i=0&user=62458XXXX248&authuser=0' \
  -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' \
  -H 'accept-language: fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7,el;q=0.6' \
  -H 'cache-control: no-cache' \
  -H 'cookie: SOCS=CAESHAgxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxiA8sq1Bg; AEC=AVYB7cohkzdixxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx; SEARCH_SAMESITE=CxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxwB; NID=518=XkUxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxRNCq9YbhL27-tRVTNO29BxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxLfy_G5G4wjkIvwk-Ar_ghOcCPuI_gmqyLsGUGWIhoRntaKwk5R-f_ZLafYuJxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxETTjkzjNjPj-_2Dk7izfPBXorZy6R7jOZopjN-G5_eN0d525avAr1paHB-6g_87yRYsdnNVkTuXdFBqO8fbGFWWGKJy9-KMbQopcFr84teITSv2U0yMq40yQ9yTjaH9RLRQxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxp2UF5wkBukB6y1AGOhDoAlB6WpcB6UUyLan-9m5chmwnOoPh7NABINcDuWiOkfJJxUTxP9xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxFRL5hdeXes4KjrgUVCFXauanLdDM1Mbg6YEgk41M03E5MQXlIpergqvG6kEU2TcCEa2uTLonOSRRqv81iiaiSEFi5tO_J5CY6bnhfYmrFHSNJCGsy-6YeL0LTxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxlrb02xWwpKGHumrFgjoBcaRzwX9-Y5RprxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxiSkUa4eh7r4_Xe3JaFVs; __Secure-1PSIDTS=sidts-CjEBQT4rXxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPO6L1gZBOigLhAEAA; __Secure-3PSIDTS=sidts-CjEBQxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxA; __Secure-ENID=23.SE=UxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxUJSGseyEJkgJwS_2eB6aIMupfYiXUI2k7c-vpiiUmSrU42xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxNdbJ7s5_1eFWg-DG8QBfV8mRRi7vzDNHFVSxGOcaOBkexjT-oJuKiFM-acxVUj3_tngw4-_J_cK7LOo9atcPdruJ87DHGkpBjSWS_5t_cHi7JQJ8hY5ttaQ3LmoQIS8HUW2Sk_a2tZeAct3XAAaVmJCgHDwBIIs1odQL8ibyJQ5nGKbRGmO4gAcYBY8W1jj6438MItVXScwDcWgHsWv-l4r5R5vxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxEJy8p-yzO3_4E2nGRULRr7xYiTwkm7hCRV_QwsSWgpeR1RoDACP5fM9dcFsbx1FtWLbiTZb1i9mZJmhJ1G6czvtwa7scV2ysOOYiRKfYI_fvqQu5n7kVWGw; SID=g.a00xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx4QkuHMIwivRopeAJp-qFc3ik0n9dxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxAUF8yKogu7VO-GjSkp9xOw9STggE0076; __Secure-1PSID=g.a0xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxK52WCtNxsDP_iAACgYKAbESARYSFQHGX2MikEuSn1BT7pN0ENdL7abNwBoVAUF8yKoJmXPPQ1mp3-lhgu_1ny3N0076; __Secure-3PSID=g.a000pwgwyR_OyPtsJIeqRLtdh5DsHn1DEy4b4SZ8x3yx4QkuHMIw8maH_lt6d-uzvfHNDThmKQACgYKAV4SARYSFQHGX2Miws1LeH7PpoLw1O3xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxu-ESV0076; HSID=Axxxxxxxxxxxxxxx0Aq; SSID=A8SApJyvoh8j0w9eZ; APISID=LrlxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPw9; SAPISID=xNgxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxEteV5tHU; __Secure-1PAPISID=xNgZxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxU; __Secure-3PAPISID=xNgxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxtHU; SIDCC=AKEyxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxMQ; __Secure-1PSIDCC=AKEyxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx3jAF5--ybeuwzq8oj5g; __Secure-3PSIDCC=AKEyXxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxMmAmWB_vZwDkACrU9EXE' \
  -H 'pragma: no-cache' \
  -H 'priority: u=0, i' \
  -H 'referer: https://takeout.google.com/' \
  -H 'sec-ch-ua: "Chromium";v="130", "Google Chrome";v="130", "Not?A_Brand";v="99"' \
  -H 'sec-ch-ua-arch: "x86"' \
  -H 'sec-ch-ua-bitness: "64"' \
  -H 'sec-ch-ua-full-version-list: "Chromium";v="130.0.6723.69", "Google Chrome";v="130.0.6723.69", "Not?A_Brand";v="99.0.0.0"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-model: ""' \
  -H 'sec-ch-ua-platform: "Linux"' \
  -H 'sec-ch-ua-platform-version: "6.1.0"' \
  -H 'sec-ch-ua-wow64: ?0' \
  -H 'sec-fetch-dest: document' \
  -H 'sec-fetch-mode: navigate' \
  -H 'sec-fetch-site: same-site' \
  -H 'upgrade-insecure-requests: 1' \
  -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36' \
  -H 'x-browser-channel: stable' \
  -H 'x-browser-copyright: Copyright 2024 Google LLC. All rights reserved.' \
  -H 'x-browser-validation: 3gQbjS+gXXXXXXXXXXXXXXXZHAA=' \
  -H 'x-browser-year: 2024' \
  -H 'x-client-data: CI+2yQEIxxxxxxxxxxxxxxxxxxxxxxxxxxHLAQid/swBCIegzQEI/aX4BCxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxJyxzgE='

Aujourd'hui, j'arrive bien à ajouter le download avec l'API mais pas les HEADERS NECESSAIRES :

le cURL qui fonctionne pour l'ajout c'est : 

curl -s "https://fbx.fbx.lan/api/v12/downloads/add" \
-H 'X-Fbx-App-Auth: yvF93xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6+jC3mcQ8mGX' \
-X POST \
--cacert /dev/shm/fbx-cacert \
--data-urlencode download_url=https://takeout-download.usercontent.google.com/download/takeout-20241101T091938Z-001.tgz?j=380XXXXX-6XX5-4XX2-9XXd-dXXXXXXXXXbf&i=0&user=62458XXXX248&authuser=0 \
--data-urlencode username=xxxxxxx@gmail.com \
--data-urlencode password=XXXXXXXXXXXXXXXXXXX \
--data-urlencode download_dir=/FBX24T/dl/ \
--data-urlencode filename=takeout_test.tgz \
--data-urlencode cookie1='SOCS=CAESHAgxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxiA8sq1Bg; AEC=AVYB7cohkzdixxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx; SEARCH_SAMESITE=CxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxwB; NID=518=XkUxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxRNCq9YbhL27-tRVTNO29BxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxLfy_G5G4wjkIvwk-Ar_ghOcCPuI_gmqyLsGUGWIhoRntaKwk5R-f_ZLafYuJxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxETTjkzjNjPj-_2Dk7izfPBXorZy6R7jOZopjN-G5_eN0d525avAr1paHB-6g_87yRYsdnNVkTuXdFBqO8fbGFWWGKJy9-KMbQopcFr84teITSv2U0yMq40yQ9yTjaH9RLRQxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxp2UF5wkBukB6y1AGOhDoAlB6WpcB6UUyLan-9m5chmwnOoPh7NABINcDuWiOkfJJxUTxP9xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxFRL5hdeXes4KjrgUVCFXauanLdDM1Mbg6YEgk41M03E5MQXlIpergqvG6kEU2TcCEa2uTLonOSRRqv81iiaiSEFi5tO_J5CY6bnhfYmrFHSNJCGsy-6YeL0LTxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxlrb02xWwpKGHumrFgjoBcaRzwX9-Y5RprxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxiSkUa4eh7r4_Xe3JaFVs; __Secure-1PSIDTS=sidts-CjEBQT4rXxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPO6L1gZBOigLhAEAA; __Secure-3PSIDTS=sidts-CjEBQxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxA; __Secure-ENID=23.SE=UxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxUJSGseyEJkgJwS_2eB6aIMupfYiXUI2k7c-vpiiUmSrU42xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxNdbJ7s5_1eFWg-DG8QBfV8mRRi7vzDNHFVSxGOcaOBkexjT-oJuKiFM-acxVUj3_tngw4-_J_cK7LOo9atcPdruJ87DHGkpBjSWS_5t_cHi7JQJ8hY5ttaQ3LmoQIS8HUW2Sk_a2tZeAct3XAAaVmJCgHDwBIIs1odQL8ibyJQ5nGKbRGmO4gAcYBY8W1jj6438MItVXScwDcWgHsWv-l4r5R5vxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxEJy8p-yzO3_4E2nGRULRr7xYiTwkm7hCRV_QwsSWgpeR1RoDACP5fM9dcFsbx1FtWLbiTZb1i9mZJmhJ1G6czvtwa7scV2ysOOYiRKfYI_fvqQu5n7kVWGw; SID=g.a00xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx4QkuHMIwivRopeAJp-qFc3ik0n9dxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxAUF8yKogu7VO-GjSkp9xOw9STggE0076; __Secure-1PSID=g.a0xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxK52WCtNxsDP_iAACgYKAbESARYSFQHGX2MikEuSn1BT7pN0ENdL7abNwBoVAUF8yKoJmXPPQ1mp3-lhgu_1ny3N0076; __Secure-3PSID=g.a000pwgwyR_OyPtsJIeqRLtdh5DsHn1DEy4b4SZ8x3yx4QkuHMIw8maH_lt6d-uzvfHNDThmKQACgYKAV4SARYSFQHGX2Miws1LeH7PpoLw1O3xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxu-ESV0076; HSID=Axxxxxxxxxxxxxxx0Aq; SSID=A8SApJyvoh8j0w9eZ; APISID=LrlxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPw9; SAPISID=xNgxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxEteV5tHU; __Secure-1PAPISID=xNgZxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxU; __Secure-3PAPISID=xNgxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxtHU; SIDCC=AKEyxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxMQ; __Secure-1PSIDCC=AKEyxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx3jAF5--ybeuwzq8oj5g; __Secure-3PSIDCC=AKEyXxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxMmAmWB_vZwDkACrU9EXE'

Le résulta de mon script de download est le suivant : 

operation completed: 
{"success":true}

result:
{"id":531}


{"success":true,"result":{"id":531}}

task 531 done ... 

Download Task log: task 531

2024-11-02 10:23:34 info: start url https://xxxxxxxxxxxx%40gmail.com:XXXXXXXXXXXX@takeout-download.usercontent.google.com/download/takeout-20241101T091938Z-001.tgz?j=38XXXXX17-6XX5-4XX2-9XXd-de84xxxxxxxf&i=0&user=62xxxxxxxxx8&authuser=0 (crawling: 1)
2024-11-02 10:23:34 dbg: host resolved to 172.217.20.193:443
2024-11-02 10:23:34 dbg: connecting to remote host...
2024-11-02 10:23:34 dbg: connected
2024-11-02 10:23:34 dbg: sending request headers:
2024-11-02 10:23:34 dbg: 	User-Agent: Mozilla/5.0
2024-11-02 10:23:34 dbg: 	Host: takeout-download.usercontent.google.com:443
2024-11-02 10:23:34 dbg: request headers sent
2024-11-02 10:23:34 dbg: got response headers:
2024-11-02 10:23:34 dbg: 	X-GUploader-UploadID: AHmUCY3fBrhctbaMFhsiTAw5SY63kG2kCZqIwzfX14yyMJZrfX0ewdnKQq9iTwQR4-bMz0Ip-IXahEF2-g
2024-11-02 10:23:34 dbg: 	Server: UploadServer
2024-11-02 10:23:34 dbg: 	Location: https://accounts.google.com/ServiceLogin?continue=https%3A%2F%2Ftakeout-download.usercontent.google.com%2Fdownload%2Ftakeout-20241101T091938Z-001.tgz%3Fj%3xxxxxxxxxxxx7-6xx5-4XXX-9XXd-deXXXXXXXXXf%26i%3D0%26user%3D6XXXXXXXXXXXX8%26authuser%3D0&authuser=0
2024-11-02 10:23:34 dbg: 	Expires: Sat, 02 Nov 2024 09:23:34 GMT
2024-11-02 10:23:34 dbg: 	Date: Sat, 02 Nov 2024 09:23:34 GMT
2024-11-02 10:23:34 dbg: 	Content-Type: text/html; charset=UTF-8
2024-11-02 10:23:34 dbg: 	Content-Security-Policy: sandbox allow-scripts
2024-11-02 10:23:34 dbg: 	Content-Security-Policy: default-src 'none'; img-src 'self'; report-uri https://csp.withgoogle.com/csp/scotty/2;
2024-11-02 10:23:34 dbg: 	Content-Length: 0
2024-11-02 10:23:34 dbg: 	Cache-Control: private, max-age=0
2024-11-02 10:23:34 dbg: 	Alt-Svc: h3=:443; ma=2592000,h3-29=:443; ma=2592000
2024-11-02 10:23:34 dbg: host resolved to 173.194.76.84:443
2024-11-02 10:23:34 dbg: connecting to remote host...
2024-11-02 10:23:34 dbg: connected
2024-11-02 10:23:34 dbg: sending request headers:
2024-11-02 10:23:34 dbg: 	User-Agent: Mozilla/5.0
2024-11-02 10:23:34 dbg: 	Host: accounts.google.com:443
2024-11-02 10:23:34 dbg: request headers sent
2024-11-02 10:23:34 dbg: got response headers:
2024-11-02 10:23:34 dbg: 	X-XSS-Protection: 0
2024-11-02 10:23:34 dbg: 	X-Content-Type-Options: nosniff
2024-11-02 10:23:34 dbg: 	Strict-Transport-Security: max-age=31536000; includeSubDomains
2024-11-02 10:23:34 dbg: 	Set-Cookie: __Host-GAPS=1:bHayjBxKs6Li-j-bHU_sQ4m2_ZSFeQ:6iNsTC2xAV5NGq-v; Expires=Mon, 02-Nov-2026 09:23:34 GMT; Path=/; Secure; HttpOnly; Priority=HIGH
2024-11-02 10:23:34 dbg: 	Server: ESF
2024-11-02 10:23:34 dbg: 	Pragma: no-cache
2024-11-02 10:23:34 dbg: 	Permissions-Policy: ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-full-version-list=*, ch-ua-model=*, ch-ua-wow64=*, ch-ua-form-factors=*, ch-ua-platform=*, ch-ua-platform-version=*
2024-11-02 10:23:34 dbg: 	Location: https%3A%2F%2Ftakeout-download.usercontent.google.com%2Fdownload%2Ftakeout-20241101T091938Z-001.tgz%3Fj%3xxxxxxxxxxxx7-6xx5-4XXX-9XXd-deXXXXXXXXXf%26i%3D0%26user%3D6XXXXXXXXXXXX8%26authuser%3D0&authuser=0&ifkv=AcMMx-evqDrTS5S7GALJ9xxxxxxxxxxxxxxxxxxxxxxxxxxxxx4smsIkEUVtr8YO020SMPon1CLtiQ
2024-11-02 10:23:34 dbg: 	Expires: Mon, 01 Jan 1990 00:00:00 GMT
2024-11-02 10:23:34 dbg: 	Date: Sat, 02 Nov 2024 09:23:34 GMT
2024-11-02 10:23:34 dbg: 	Cross-Origin-Resource-Policy: cross-origin
2024-11-02 10:23:34 dbg: 	Cross-Origin-Opener-Policy: unsafe-none
2024-11-02 10:23:34 dbg: 	Content-Type: application/binary
2024-11-02 10:23:34 dbg: 	Content-Security-Policy: script-src 'report-sample' 'nonce-wimNlo949vvqyB32Q-0ScA' 'unsafe-inline';object-src 'none';base-uri 'self';report-uri /_/AccountsSigninPassiveLoginHttp/cspreport;worker-src 'self'
2024-11-02 10:23:34 dbg: 	Content-Security-Policy: require-trusted-types-for 'script';report-uri /_/AccountsSigninPassiveLoginHttp/cspreport
2024-11-02 10:23:34 dbg: 	Content-Length: 0
2024-11-02 10:23:34 dbg: 	Cache-Control: no-cache, no-store, max-age=0, must-revalidate
2024-11-02 10:23:34 dbg: 	Alt-Svc: h3=:443; ma=2592000,h3-29=:443; ma=2592000
2024-11-02 10:23:34 dbg: 	Accept-CH: Sec-CH-UA-Arch, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Model, Sec-CH-UA-WoW64, Sec-CH-UA-Form-Factors, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version
2024-11-02 10:23:34 dbg: host resolved to 108.177.15.84:443
2024-11-02 10:23:34 dbg: connecting to remote host...
2024-11-02 10:23:34 dbg: connected
2024-11-02 10:23:34 dbg: sending request headers:
2024-11-02 10:23:34 dbg: 	User-Agent: Mozilla/5.0
2024-11-02 10:23:34 dbg: 	Host: accounts.google.com:443
2024-11-02 10:23:34 dbg: request headers sent
2024-11-02 10:23:34 dbg: got response headers:
2024-11-02 10:23:34 dbg: 	X-XSS-Protection: 1; mode=block
2024-11-02 10:23:34 dbg: 	X-Frame-Options: DENY
2024-11-02 10:23:34 dbg: 	X-Content-Type-Options: nosniff
2024-11-02 10:23:34 dbg: 	Strict-Transport-Security: max-age=31536000; includeSubDomains
2024-11-02 10:23:34 dbg: 	Set-Cookie: __Host-GAPS=1:BHfoVa3aqa22JXPLNKZCOEUJNFFGHA:iyuMGzT7CKMtNr9w;Path=/;Expires=Mon, 02-Nov-2026 09:23:34 GMT;Secure;HttpOnly;Priority=HIGH
2024-11-02 10:23:34 dbg: 	Server: GSE
2024-11-02 10:23:34 dbg: 	Report-To: {group:

Sucessfully delete task #531: {"success":true}

Et le fichier téléchargé est en fait une page HTML avec pas mal de JS dont 1 JS contenant un gros JSON pour les country … car en fait ce qui est download c'est la page de login Google Workspace ! et non le fichier Google Takeout
voici le fichier ouvert dans firefox (link valable 30j):
https://transfert.free.fr/RYrf3qK

Voici ce que dit la DOC de l'API (et d'après mes tests FreeboxOS utilise également l'API) : 

POST /api/v8/downloads/add

Parameters
download_url (string) – The URL
download_url_list (string) – A list of URL separated by a new line delimiter (use download_url or download_url_list)
download_dir (string) – The download destination directory (optional: will use the configuration download_dir by default)
filename (string) – Override the name of the destination file. Only valid with one, non-recursive download_url.
hash (string) – Verify the hash of the downloaded file. The format is sha256:xxxxxx or sha512:xxxxxx; or the URL of a SHA256SUMS, SHA512SUMS, -CHECKSUM or .sha256 file. Only valid with one, non-recursive download_url.
recursive (bool) – If true the download will be recursive
username (string) – Auth username (optional)
password (string) – Auth password (optional)
archive_password (string) – The password required to extract downloaded content (only relevant for nzb)
cookies (string) – The http cookies (to be able to pass session cookies along with url). This is the content of the HTTP Cookie header, for example: cookie1=value1; cookie2=value2

NOTE: instead of passing password and username you can include them in the URL.

Example request : Single download add:

POST /api/v8/downloads/add HTTP/1.1
Host: mafreebox.freebox.fr

download_url=http%3A%2F%2Fcdimage.debian.org%2Fdebian-cd%2F6.0.6%2Famd64%2Fbt-cd%2Fdebian-6.0.6-amd64-CD-1.iso.torrent
&download_dir=L0Rpc3F1ZSBkdXIvVMOpbMOpY2hhcmdlbWVudHMv

Donc comme vous le voyez, l'API et FreeboxOS ne permettent pas de configurer les headers nécessaires à la communication avec les serveurs de GOOGLE (ils font bien c… d'ailleurs ces Google ou autre Microsoft, Dropbox , etc avec le javascript partout ! Une url sèche après auth type SAML de la session Windows ou Linux complète serait beaucoup plus pratique… Bref, il en est autrement)


Il faudrait rajouter la possibilité de passer des 'CUSTOM HEADERS' pour passer les Headers suivants : 

  -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' \
  -H 'accept-language: fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7,el;q=0.6' \
  -H 'cache-control: no-cache' \
  -H 'pragma: no-cache' \
  -H 'priority: u=0, i' \
  -H 'referer: https://takeout.google.com/' \
  -H 'sec-ch-ua: "Chromium";v="130", "Google Chrome";v="130", "Not?A_Brand";v="99"' \
  -H 'sec-ch-ua-arch: "x86"' \
  -H 'sec-ch-ua-bitness: "64"' \
  -H 'sec-ch-ua-full-version-list: "Chromium";v="130.0.6723.69", "Google Chrome";v="130.0.6723.69", "Not?A_Brand";v="99.0.0.0"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-model: ""' \
  -H 'sec-ch-ua-platform: "Linux"' \
  -H 'sec-ch-ua-platform-version: "6.1.0"' \
  -H 'sec-ch-ua-wow64: ?0' \
  -H 'sec-fetch-dest: document' \
  -H 'sec-fetch-mode: navigate' \
  -H 'sec-fetch-site: same-site' \
  -H 'upgrade-insecure-requests: 1' \
  -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36' \
  -H 'x-browser-channel: stable' \
  -H 'x-browser-copyright: Copyright 2024 Google LLC. All rights reserved.' \
  -H 'x-browser-validation: 3gQbjS+gXXXXXXXXXXXXXXXZHAA=' \
  -H 'x-browser-year: 2024' \
  -H 'x-client-data: CI+2yQEIxxxxxxxxxxxxxxxxxxxxxxxxxxHLAQid/swBCIegzQEI/aX4BCxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxJyxzgE='

Donc j'insiste le mieux serait que soit géré le processus complet de la génération de liens Google Takeout puis la session connectée à Google puis l'ajout des headers manquant pour permettre le download des liens Google Takeout par la seedbox.

j'ai bien peur que ce soit très très compliqué autrement de récupérer les 19 cookies : 

SOCS
AEC
SEARCH_SAMESITE
NID
__Secure-1PSIDTS
__Secure-3PSIDTS
__Secure-ENID
SID
__Secure-1PSID
__Secure-3PSID
HSID
SSID
APISID
SAPISID
__Secure-1PAPISID
__Secure-3PAPISID
SIDCC
__Secure-1PSIDCC
__Secure-3PSIDCC

De même, ces ajouts s'ils sont fait sous formes de nouveaux paramètres API permettront certainement de télécharger des archives SHAREPOINT (idem, même idée, autommatiser les backup de son SHAREPOINT sur le NAS de la FREEBOX, idem pour DROPBOX ou tout autre fournisseur fou de javascript autogénérant du HTML …)

Mon point de vue perso : franchement pour pomper un fichier sur un FTP en explicite TLS (niveau de sécurité fort), il n'y a pas besoin de tout ce code JS ni de générer du HTML dynamiquement ou de recheck 300 fois l'authentification, la session , etc… Google et les autres abusent, car au final, on fait la même chose, à savoir pomper un fichier sur un serveur en TLS et de manière authentifiée !

En vous remerciant d'avance
Cordialement
nbanba

nbanba commented on 02.11.2024 17:38

Bonjour

Alors en faisant un truc bien cochon (ci après…), j'arrive à passer une bonne partie des headers à la seedbox (la seedbox utilise la libcurl ?): 

curl -s https://fbx.fbx.lan/api/v12/downloads/add \
-H "X-Fbx-App-Auth: $_SESSION_TOKEN" -X POST --cacert /dev/shm/fbx-cacert" \
--data-urlencode download_url='https://takeout-download.usercontent.google.com/download/takeout-20241102T164012Z-001.tgz?j=46xxxxxxxf6xxxxxxxxxxxxxxxxxxxxa9205&i=0&user=62xxxxxxxxx8&authuser=0' \
--data-urlencode username=xxxxxxxxxx@gmail.com \
--data-urlencode password=xxxxxxxxxxxxxx \
--data-urlencode download_dir=/FBX24T/dl/ \
--data-urlencode filename=takeout_testZZ.tgz \
--data-urlencode cookie="SOCS=CAxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxA8sq1Bg;<<17_AUTRES_COOKIE__;__Secure-3PSIDCC=AKEyXzUBxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxMQ;\
-H'accept:%20text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7';\
-H'accept-language:%20fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7,el;q=0.6';\
-H'cache-control:%20no-cache';\
-H'pragma:%20no-cache';\
-H'priority:%20u=0,%20i';\
-H'referer:%20https://takeout.google.com/';\
-H'sec-ch-ua:%20Chromium;v=130,%20Google%20Chrome;v=130,%20Not?A_Brand;v=99';\
-H'sec-ch-ua-mobile:%20?0';\
-H'sec-ch-ua-platform:%20Linux';\
-H'sec-fetch-dest:%20document';\
-H'sec-fetch-mode:%20navigate';\
-H'sec-fetch-site:%20same-site';\
-H'upgrade-insecure-requests:%201';\
-H'user-agent:%20Mozilla/5.0%20%28X11;%20Linux%20x86_64%29%20AppleWebKit/537.36%20%28KHTML,%20like%20Gecko%29%20Chrome/130.0.0.0%20Safari/537.36';\
-H'x-browser-channel:%20stable';\
-H'x-browser-copyright:%20Copyright%202024%20Google%20LLC.%20All%20rights%20reserved.';\
-H'x-browser-validation:%203gQbjS+gxxxxxxxxxxxxxxxxxxA=';\
-H'x-browser-year:%202024';\
-H'x-client-data:%20CI+2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxzgE='"

mais ça ne fonctionne toujours pas malgré tous les headers de policy de sécurité correctement reçu visiblement d'après les logs de la seedbox : 

2024-11-02 18:03:12 dbg: host resolved to 142.250.201.161:443
2024-11-02 18:03:12 dbg: connecting to remote host...
2024-11-02 18:03:12 dbg: connected
2024-11-02 18:03:12 dbg: sending request headers:
2024-11-02 18:03:12 dbg: 	User-Agent: Mozilla/5.0
2024-11-02 18:03:12 dbg: 	Host: takeout-download.usercontent.google.com:443
2024-11-02 18:03:12 dbg: request headers sent
2024-11-02 18:03:12 dbg: got response headers:
2024-11-02 18:03:12 dbg: 	X-GUploader-UploadID: AHmUCY3HhxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxMPd-iUA
2024-11-02 18:03:12 dbg: 	Server: UploadServer
2024-11-02 18:03:12 dbg: 	Location: https://accounts.google.com/ServiceLogin?continue=https%3A%2F%2Ftakeout-download.usercontent.google.com%2Fdownload%2Ftakeout-20241102T164012Z-001.tgz%3Fj%3xxxxxx907-fxxd-4xxf-axxd-b5xxxxxxxxxxxxxi%3D0%26user%3Dxxxxxxxxxxx8%26authuser%3D0&authuser=0
2024-11-02 18:03:12 dbg: 	Expires: Sat, 02 Nov 2024 17:03:12 GMT
2024-11-02 18:03:12 dbg: 	Date: Sat, 02 Nov 2024 17:03:12 GMT
2024-11-02 18:03:12 dbg: 	Content-Type: text/html; charset=UTF-8
2024-11-02 18:03:12 dbg: 	Content-Security-Policy: sandbox allow-scripts
2024-11-02 18:03:12 dbg: 	Content-Security-Policy: default-src 'none'; img-src 'self'; report-uri https://csp.withgoogle.com/csp/scotty/2;
2024-11-02 18:03:12 dbg: 	Content-Length: 0
2024-11-02 18:03:12 dbg: 	Cache-Control: private, max-age=0
2024-11-02 18:03:12 dbg: 	Alt-Svc: h3=:443; ma=2592000,h3-29=:443; ma=2592000
2024-11-02 18:03:12 dbg: host resolved to 64.233.166.84:443
2024-11-02 18:03:12 dbg: connecting to remote host...
2024-11-02 18:03:12 dbg: connected
2024-11-02 18:03:12 dbg: sending request headers:
2024-11-02 18:03:12 dbg: 	User-Agent: Mozilla/5.0
2024-11-02 18:03:12 dbg: 	Host: accounts.google.com:443
2024-11-02 18:03:12 dbg: request headers sent
2024-11-02 18:03:12 dbg: got response headers:
2024-11-02 18:03:12 dbg: 	X-XSS-Protection: 0
2024-11-02 18:03:12 dbg: 	X-Content-Type-Options: nosniff
2024-11-02 18:03:12 dbg: 	Strict-Transport-Security: max-age=31536000; includeSubDomains
2024-11-02 18:03:12 dbg: 	Set-Cookie: __Host-GAPS=1:DI7dcpP_PNYRsWl_7YoMFOsD99XHdQ:FXaznnBZmzskoS7S; Expires=Mon, 02-Nov-2026 17:03:12 GMT; Path=/; Secure; HttpOnly; Priority=HIGH
2024-11-02 18:03:12 dbg: 	Server: ESF
2024-11-02 18:03:12 dbg: 	Pragma: no-cache
2024-11-02 18:03:12 dbg: 	Permissions-Policy: ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-full-version-list=*, ch-ua-model=*, ch-ua-wow64=*, ch-ua-form-factors=*, ch-ua-platform=*, ch-ua-platform-version=*
2024-11-02 18:03:12 dbg: 	Location: https://accounts.google.com/InteractiveLogin?authuser=0&continue=https://takeout-download.usercontent.google.com/download/takeout-20241102T164012Z-001.tgz?j%3Dxxxxxx07-fxxd-4xxf-axxd-bxxxxxxxxxxxxxxxxxxx%26user%3D6xxxxxxxxxx8%26authuser%3D0&ifkv=AcxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxNWg
2024-11-02 18:03:12 dbg: 	Expires: Mon, 01 Jan 1990 00:00:00 GMT
2024-11-02 18:03:12 dbg: 	Date: Sat, 02 Nov 2024 17:03:12 GMT
2024-11-02 18:03:12 dbg: 	Cross-Origin-Resource-Policy: cross-origin
2024-11-02 18:03:12 dbg: 	Cross-Origin-Opener-Policy: unsafe-none
2024-11-02 18:03:12 dbg: 	Content-Type: application/binary

2024-11-02 18:03:12 dbg: Content-Security-Policy: require-trusted-types-for 'script';report-uri /_/AccountsSigninPassiveLoginHttp/cspreport 2024-11-02 18:03:12 dbg: Content-Security-Policy: script-src 'report-sample' 'nonce-jkxxxxxxxxxxxxxxxxxxAQ' 'unsafe-inline';object-src 'none';base-uri 'self';report-uri /_/AccountsSigninPassiveLoginHttp/cspreport;worker-src 'self' 

2024-11-02 18:03:12 dbg: 	Content-Length: 0

2024-11-02 18:03:12 dbg: Cache-Control: no-cache, no-store, max-age=0, must-revalidate

2024-11-02 18:03:12 dbg: Alt-Svc: h3=:443; ma=2592000,h3-29=:443; ma=2592000 2024-11-02 18:03:12 dbg: Accept-CH: Sec-CH-UA-Arch, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Model, Sec-CH-UA-WoW64, Sec-CH-UA-Form-Factors, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version 

2024-11-02 18:03:12 dbg: host resolved to 64.233.166.84:443
2024-11-02 18:03:12 dbg: connecting to remote host...
2024-11-02 18:03:12 dbg: connected
2024-11-02 18:03:12 dbg: sending request headers:
2024-11-02 18:03:12 dbg: 	User-Agent: Mozilla/5.0
2024-11-02 18:03:12 dbg: 	Host: accounts.google.com:443
2024-11-02 18:03:12 dbg: request headers sent
2024-11-02 18:03:12 dbg: got response headers:
2024-11-02 18:03:12 dbg: 	X-XSS-Protection: 1; mode=block

2024-11-02 18:03:12 dbg: X-Frame-Options: DENY 

2024-11-02 18:03:12 dbg: 	X-Content-Type-Options: nosniff
2024-11-02 18:03:12 dbg: 	Strict-Transport-Security: max-age=31536000; includeSubDomains
2024-11-02 18:03:12 dbg: 	Set-Cookie: __Host-GAPS=1:lLTsrbpPHeHZDY_qf4VJDip8v80NeA:5p6BaT6K3p4ywkYV;Path=/;Expires=Mon, 02-Nov-2026 17:03:12 GMT;Secure;HttpOnly;Priority=HIGH
2024-11-02 18:03:12 dbg: 	Server: GSE
2024-11-02 18:03:12 dbg: 	Report-To: {group:

Donc il doit y avoir encore un HEADER de mal interprété
⇒ Ce serait bien d'ajouter ce type de fonctions (au moins des champs fait pour passer les HEADERS, et comme pour les COOKIES, il fait pouvoir en passer beaucoup (plus de 25 parfois…)

En vous remerciant d'avance
Cordialement
nbanba

Loading...

Available keyboard shortcuts

Tasklist

Task Details

Task Editing