Dropbox is vile and I want no part of it in my life. However, one particular tool I work with has no way of getting its data to me except via Dropbox, and I can't fix that.
I have one of those anonymous "share" links for the folder in which my data materializes (the "dropbox.
So I tried doing it the "official" way and creating and "app" and generating an OAuth token and using the official API, but after successfully authenticating, all it ever says is "missing_scope, required_scope: files.metadata.read". (That checkbox is among the many that are checked on my app's config page.)
Have any of you ever successfully used this API to list a folder and read a file?
Note: I am not asking for your anecdote about how you installed the Dropbox GUI on your machine. And even if it is possible to install a Dropbox file-system-impersonating rootkit daemon on a headless CentOS 7 server, I have no interest in doing that. I just want to mirror files out of this one URL, by polling from cron.
so, I'm woefully not very knowledgeable in this stuff, but my wife wants to mirror some stuff to cloud storage so I had a look at rclone. Maybe it would make this all a lot less painful?
https://rclone.org/docs/
I'll go on a limb here and ask a very stupid question. Have you tried using rclone, https://rclone.org/dropbox/?
No, because it sounded like it was just an over-complicated rsync clone, that required me to set up a Dropbox app anyway?
Have you successfully used rclone to mirror arbitrary Dropbox folders? Because the comment below suggest that the API just doesn't support that.
I won't know about arbitrary folders, but works for my personal account. rclone offers "
--dropbox-shared-folders", which might work on your use case.
Why are Dropbox vile? Facebook/Twitter vile or technically vile?
Don't take this as gospel because I only futzed around with this enough to get this to work for a personal blog thing (& I'm downloading a zip of the entire folder, not listing files), but I _believe_ any given app is limited to a single "application" folder without general access to the toplevel dropbox folder. E.g., I have code here [1] to download a "/blog" directory. However, that's mounted at `~/Dropbox/Apps/<my app name>/blog`.
Hopefully this helps / commiserations!
[1]: https://github.com/chrisdickinson/neversaw.us/blob/latest/bin/download_folder.sh
Well, if apps are sandboxed like that, that would definitely explain it. The docs sure didn't make that clear.
You specify upon app creation whether you want your app to be sandboxed or not: https://www.dropbox.com/developers/reference/developer-guide (See “scoping”).
I can confirm that the API works (apart from nasty behavior for non-ASCII file names), when invoked. I believe the error you get suggests that you did not specify the scope parameter in you oauth2/authorize request: https://www.dropbox.com/developers/documentation/http/documentation
I just want to commiserate: dropbox is indeed vile and they have not done a single fucking thing in the past decade to make any aspect of any experience they offer better in any dimension
how about one of the third party clients? I don’t know about Linux, but there are decent Mac ones.
I feel sorry for you
I have used rclone in the past to grab folders.
The official dropbox headless (!) linux binary also does the job, and it allows you to exclude folders, so you could narrow it down to hold just a local copy of the specific folder you want.
No package install needed, it's a binary that runs from your homedir and is controlled by a python script.
If you're anxious about telemetry, you can launch it, check for the "sync" status, then shut it down again when it's done.
check the last paragraph on this page: https://www.dropbox.com/install-linux
Don't use the DEB or RPM packages, just the daemon. You can check the install script yourself, it just puts it in .dropboxd
It is what I'm currently using to make a backup of my family's shared dropbox content. I rsync the shared folder periodically to a safe, versioned location.
Everyone suggesting "here's a 3rd party reimplementation of the Dropbox daemon and GUI that populates ~/Dropbox/" is missing the fact that Dropbox share links, the entire point of this exercise, don't show up there at all.
sorry, I mistakenly thought it was from your own account. A possible workaround: create a dummy Dropbox account and force the other party to share with you? Probably also not feasible, since it sounds like the tool is not in your control :(
Other idea is using Selenium to manipulate a spun up headless browser to trigger the zip download
maestral was pretty inoffensive when I last used it https://www.maestral.app/ could be less work than doing a cutdown client?
Yes, and I initially hit the same error you're seeing. I updated the permissions, same error.
Then I re-created the Token and all was well. I imagine you've already tried this, but since you didn't explicitly mention the order of permissions-vs-token-creation, it was at least worth mentioning.
Disclaimer. I have never used Dropbox. However, I am keenly aware of Oauth2 fuckery. It's a mess.
You don't actually mention what scopes you specified in the authorize call, soooo.... when calling /oauth/authorize to get the token, did you include this "files.metadata.read" scope in the scope parameter? If not, the token it gives you won't let the API know you're allowed. You will get a valid token. It will not just give you all the permissions your app is allowed to request. It will only give you permissions you're allowed and you ask for. Permissions for what you're allowed to do are signed right into the token, and if you don't ask, you dont get.
"I am this app, I would like a token to do NOTHING, please" -> "OK, here you go buddy! Success!"
"Hello API, I have this token..." -> "FUCK OFF"
You need to say "I am this app, I would like a token that lets me do THING_I_WANT, please". Add a scope.
I think you're right, I needed to re-issue my session token after clicking the boxes.
After doing that I'm able to get metadata on things in my own dropbox account with e.g. {"path":"/thing.png"} but I still can't figure out how to refer to files that exist outside of my own folder like the /sh/ link of interest.
While {"path":"id:XXXXXXXXX"} works on my own file, I was able to find the "file_id" of a file within the /sh/ tree, and I cannot access that file by id.
Continuing to suspect that the Dropbox API only lets you touch your own copies of files, not files shared with you from someone else's account.
I have now looked at the dropbox API site, which, again, is the most I have ever used dropbox.
https://developers.dropbox.com/dbx-sharing-guide
> You can use a shared link to retrieve metadata by calling /sharing/get_shared_link_metadata
In that example the shared link being queried is of the format "/s/". In the overall API document, the only links with "/sh/" are seen in the output of /create_shared_link_with_settings, /get_shared_link_file, /get_shared_link_metadata and /modify_shared_link_settings, so I'm hoping that "/s/" and "/sh/" are just different versions of the same conceptual thing, a shared link, and not entirely different concepts and somehow /get_shared_link_metadata won't work with your "/sh/" URL
Thanks, I'm making progress on this. It's even more horrible than OAuth things usually are. (E.g. shared folders have both "folder IDs" and "folder IDs" and they are not the same thing.)
Anyway, my current struggle is, how do I either get a long-lived access token, or refresh the one that I have? The API seems to want a token before I can refresh it.
Poking around, it seems that this will have been the fourth time in my life that I have had to implement OAuth token refreshing, and each of those sites does it a completely different way.
Ask for the authorization code with token_access_type=offline (the incantaion needed to get one of these varies between providers because the standard doesn't make them do it uniformly, this appears to be the dropbox incantation), then call /oauth2/token?grant_type=authorization_code&code=..., and it will give you _two_ codes in the response.
One is access_token, short lived, you can use now.
The other is refresh_token, one-time-use, but you don't have to use for a long time (for some providers, you can go a year or more without using it and it'll still be valid. They can change it at any time. They're meant to say in their documentation. The API itself will never say, it only says how long the access token lasts).
When the access_token expires, call /oauth2/token?grant_type=refresh_token&refresh_token=... with the refresh token, and it'll issue you a new access_token and refresh_token. When that access token expires, use the new refresh token, and so on.
I still can't tell how I'm supposed to do that. Loading /oauth2/authorize?client_id=APP_ID&token_access_type=offline&response_type=code in a browser makes me click authorize and then says "enter this code in the app" and returns a 43 character code that does not work as an access token. The access tokens that worked were 143 characters.
Posting that to "/oauth2/token grant_type=refresh_token refresh_token=...that... client_id=... client_secret=..." says "refresh token is malformed".
You call /oauth2/authorize like that. The code is your authorization code, let's say it's 12345. I'm suprised it lets you call without a callback url, but hooray that it does.
You then must POST /oauth2/token grant_type=authorization_code code=12345 client_id=... client_secret=... Because you asked for offline in the authorization call, you should get a response with at least access_token and refresh_token. eg. {"access_token":"HUGALUGA", "refresh_token":HERPDERP"}
You now use the access_token on all the other API calls. Authorization: Bearer HUGALUGA
When that expires, you need another access_token. POST /oauth2/token grant_type=refresh_token refresh_token=HERPDERP client_id=... client_secret=... and it should issue another access_token. Keep doing that.
Me: and you'll get a new refresh token with each refresh
Dropbox: This request won't return a new refresh token since refresh tokens don't expire automatically and can be reused repeatedly
Well so much for goddamn standards. It looks like you get to reuse the same refresh token indefinitely each time you need a new access token (until they feel like expiring the refresh token, at which point you'll need to login again with authorize)
You have to pass token_access_type=offline to the /oath/authorize request to get a refresh token from the /oauth/token request. Otherwise you have to go through the user interactive flow every time.
I've been quite enjoying Cyberduck as an FTP-style GUI for Dropbox and Google Drive. There's also Mountain Duck if you want to interact with them as mounted network volumes.
I have also been forced to use Dropbox for one specific use case of mine, for some time now. And I'm pretty happy using https://maestral.app/.
I also thought about this, since they have a robust command line interface... maybe that could abstract away the whole authentication flow mess, but it sounds like jwz is already pretty deep into it now...
Nobody has provided any evidence that some other app's supposed command line utilities would make any of this easier. Or even that they work at all with shared links. Plus, all of them require me to do all of this OAuth crap first. So, this is the "I googled a thing, maybe you should burn it all down and start from scratch" answer.
Just out of curiosity, what don't you like about DB?