Getting Google Drive APK access_token programatically - google-drive-api

I have a linux OS based custom embedded system. There are cameras connected to my system and i want to use google drive as cloud storage for records that were taken from my cameras.
For that purpose i used almost all google drive api functions successfully by following google drive api rest documentation in
https://developers.google.com/drive/v3/web/about-sdk
Restful api functions require an access_token for authentication purpose in accordance with OAuth2 protocol in HTTP requests. Getting this token requires some one time manual operations.
1- When signed in to google with my account, i had to create a client_id and a client_secure using google developer console.
2- Then i use these credentials to get access_code. In a response to a http request google sends me a url and a grant code.
3- Then i visit the url using a browser, typing grant code and allowing for authentication manually. Then i get access_token and refresh_token with another HTTP request.
4- After that point i can use any api function successfully with given access_token. (and i refresh it if it expires using refresh_token)
My problem is one time manual operations that are made to get tokens. I am making these operation in my computer so making them in my embedded system looks very difficult. But i want to make all the steps (signing in, client_id and client_secure, visiting url, typing grant code and allowing for authentication) in my embedded system.
So, I wonder if it is possible doing all this one time manual process to get access_code without using developer console and a browser? Can i make them programatically?
This is what i tried following oauth2 google official documentation.
After creating client_id and client_Secret i simply used curl for http operations. I wrote following bash code for testing purpose. So It is not self confident and requires some files which can be empty at first and named "access_token", "folder_id", "refresh_token", "myFile" files in the same directory.
#!/bin/bash
# Google Drive API
get_file_id() {
RESPONSE=`curl --silent -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files?q=title+contains+\'$1\'\&fields=items%2Fid`
FILE_ID=`echo $RESPONSE | python -mjson.tool | grep -oP 'id"\s*:\s*"\K(.*)"' | sed 's/"//'`
}
set -e
CLIENT_SECRET="my client_secret"
CLIENT_ID="my_client_id"
BOUNDARY="foo_bar_baz"
SCOPE="https://docs.google.com/feeds"
MIME_TYPE="application/octet-stream"
ACCESS_TOKEN=`cat access_token`
REFRESH_TOKEN=`cat refresh_token`
FOLDER_ID=`cat folder_id`
if [ "$1" == "create_token" ]; then # Usage: <"create_token">
RESPONSE=`curl --silent "https://accounts.google.com/o/oauth2/device/code" -d "client_id=$CLIENT_ID&scope=$SCOPE"`
DEVICE_CODE=`echo "$RESPONSE" | python -mjson.tool | grep -oP 'device_code"\s*:\s*"\K(.*)"' | sed 's/"//'`
USER_CODE=`echo "$RESPONSE" | python -mjson.tool | grep -oP 'user_code"\s*:\s*"\K(.*)"' | sed 's/"//'`
URL=`echo "$RESPONSE" | python -mjson.tool | grep -oP 'verification_url"\s*:\s*"\K(.*)"' | sed 's/"//'`
echo -n "Go to $URL and enter $USER_CODE to grant access to this application. Hit enter when done..."
read
RESPONSE=`curl --silent "https://accounts.google.com/o/oauth2/token" -d "client_id=$CLIENT_ID&client_secret=$CLIENT_SECRET&code=$DEVICE_CODE&grant_type=http://oauth.net/grant_type/device/1.0"`
ACCESS_TOKEN=`echo "$RESPONSE" | python -mjson.tool | grep -oP 'access_token"\s*:\s*"\K(.*)"' | sed 's/"//'`
REFRESH_TOKEN=`echo "$RESPONSE" | python -mjson.tool | grep -oP 'refresh_token"\s*:\s*"\K(.*)"' | sed 's/"//'`
echo "Access Token: $ACCESS_TOKEN"
echo "Refresh Token: $REFRESH_TOKEN"
echo "$ACCESS_TOKEN" > access_token
echo "$REFRESH_TOKEN" > refresh_token
elif [ "$1" == "refresh_token" ]; then # Usage: <"refresh_token">
RESPONSE=`curl --silent "https://accounts.google.com/o/oauth2/token" --data "client_id=$CLIENT_ID&client_secret=$CLIENT_SECRET&refresh_token=$REFRESH_TOKEN&grant_type=refresh_token"`
ACCESS_TOKEN=`echo $RESPONSE | python -mjson.tool | grep -oP 'access_token"\s*:\s*"\K(.*)"' | sed 's/"//'`
echo "Access Token: $ACCESS_TOKEN"
echo "$ACCESS_TOKEN" > access_token
elif [ "$1" == "create_folder" ]; then # Usage: <"create_folder">
FOLDER_NAME=`date "+%F-%T"`
( echo -en "{ \"title\": \"$FOLDER_NAME\", \"mimeType\": \"application/vnd.google-apps.folder\" }\n" ) \
| curl -H 'GData-Version: 3.0' -v "https://www.googleapis.com/drive/v2/files" \
--header "Authorization: Bearer $ACCESS_TOKEN" \
--header "Content-Type: application/json" \
--data-binary "#-"
#save FILE_ID to filde
get_file_id $FOLDER_NAME
echo "$FILE_ID" > folder_id
elif [ "$1" == "upload_file" ]; then # Usage: <"upload_file"> <file name>
( echo -en "--$BOUNDARY\nContent-Type: application/json; charset=UTF-8\n\n{ \"title\": \"$2\", \"parents\": [ { \"id\": \"$FOLDER_ID\" } ] }\n\n--$BOUNDARY\nContent-Type: $MIME_TYPE\n\n" \
&& cat $2 && echo -en "\n\n--$BOUNDARY--\n" ) \
| curl -H 'GData-Version: 3.0' -v "https://www.googleapis.com/upload/drive/v2/files/?uploadType=multipart" \
--header "Authorization: Bearer $ACCESS_TOKEN" \
--header "Content-Type: multipart/related; boundary=\"$BOUNDARY\"" \
--data-binary "#-"
elif [ "$1" == "list_files" ]; then # Usage: <"list_files"> <number of files>
curl -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files?maxResults=$2
elif [ "$1" == "download_file" ]; then # Usage: <"download_file"> <file name>
get_file_id $2
curl -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files/$FILE_ID?alt=media
elif [ "$1" == "get_file" ]; then # Usage: <"get_file"> <file name>
get_file_id $2
curl -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files/$FILE_ID
elif [ "$1" == "delete_file" ]; then # Usage: <"delete_file"> <file name>
get_file_id $2
curl -X Delete -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files/$FILE_ID
elif [ "$1" == "trash_file" ]; then # Usage: <"trash_file"> <file name>
get_file_id $2
curl -d -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files/$FILE_ID/trash
elif [ "$1" == "untrash_file" ]; then # Usage: <"untrash_file"> <file name>
get_file_id $2
curl -d -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files/$FILE_ID/untrash
fi
exit 0
Regards

One way to do this would be to use Domain-Wide Delegation of Authority, a service account. This method works with a JWT for the authentication. The process is explained here: Using OAuth 2.0 for Server to Server Applications
The app would use the service account to authenticate as the id that you want t use to store the drive files.

Related

How to amend a bash script so it downloads the API responses locally

I have a script:
#!/bin/bash
group=("group1" "group12" "group123" "group1234" "group12345" "group123456")
for group in "${group[#]}"
do
curl -v GET https://websitehere.com/api/v2/groups/${group}/details -H "Authorization: Bearer PANDA TOKEN HERE"
done
I want to amend the script so I can download the API responses locally. CSV or JSON.
Use -o, --output <file> : Write to file instead of stdout :
#!/bin/bash
group=("group1" "group12" "group123" "group1234" "group12345" "group123456")
for group in "${group[#]}"
do
curl -v GET https://websitehere.com/api/v2/groups/${group}/details -H "Authorization: Bearer PANDA TOKEN HERE" -o ${group}.json
done
Or CSV you can use jq and > operator to redirect the output to a file :
curl -v GET https://websitehere.com/api/v2/groups/${group}/details -H "Authorization: Bearer PANDA TOKEN HERE" | jq -r '.[] | #csv' > ${group}.csv

How to specify # in username and password for curl command?

$ curl -k -i -X POST -d '{ "path" : "/" }' https://abc#example.com:qwer##53096wrxgcg.ibmaspera.com:33001/files/workspaces/42796/all/12345:2
results in:
curl: (3) Port number ended with 'I'
Without #:
$ curl -k -i -X POST -d '{ "path" : "/" }' https://abcexample.com:qwer#53096wrxgcg.ibmaspera.com:33001/files/workspaces/42796/all/12345:2
results in:
curl: (7) Failed to connect to 53096wrxgcg.ibmaspera.com port 33001: Connection timed out
How to specify # in username and password for curl command?
If the auth username or password contain an # symbol, it should be urlencoded to %40.
Your curl command could look like either of these examples:
curl -k -i -X POST -d '{ "path" : "/" }' \
https://abc%40example.com:qwer%40#53096wrxgcg.ibmaspera.com:33001/files/workspaces/42796/all/12345:2
or
curl --user abc%40example.com:qwer%40 -k -i ...

Forge request fails on postman and curl

Hello i'm trying the introduction to the design automation ofr inventor but i'm stuck on the upload parameters.
this is the article:
https://forge.autodesk.com/blog/simple-introduction-design-automation-inventor
my last curl request:
curl -X POST \
https://dasprod-store.s3.amazonaws.com \
-H 'content-type: application/octet-stream' \
-F key=apps/Hf3jB7SzAGmZnHhBdRFvVHJEaCa7xPzN/ThumbnailBundle/1 \
-F policy=eyJleHBpcmF0aW9uIjoiMjAxOS0wMi0xMlQxNzowNDozMi43NTIxNDc0WiIsImNvbmRpdGlvbnMiOlt7ImtleSI6ImFwcHMvSGYzakI3U3pBR21abkhoQmRSRnZWSEpFYUNhN3hQek4vVGh1bWJuYWlsQnVuZGxlLzEifSx7ImJ1Y2tldCI6ImRhc3Byb2Qtc3RvcmUifSx7InN1Y2Nlc3NfYWN0aW9uX3N0YXR1cyI6IjIwMCJ9LFsic3RhcnRzLXdpdGgiLCIkc3VjY2Vzc19hY3Rpb25fcmVkaXJlY3QiLCIiXSxbInN0YXJ0cy13aXRoIiwiJGNvbnRlbnQtVHlwZSIsImFwcGxpY2F0aW9uL29jdGV0LXN0cmVhbSJdLHsieC1hbXotc2VydmVyLXNpZGUtZW5jcnlwdGlvbiI6IkFFUzI1NiJ9LFsiY29udGVudC1sZW5ndGgtcmFuZ2UiLCIwIiwiMTA0ODU3NjAwIl0seyJ4LWFtei1jcmVkZW50aWFsIjoiQVNJQVRHVkpaS00zQUtFWk9ER0gvMjAxOTAyMTIvdXMtZWFzdC0xL3MzL2F3czRfcmVxdWVzdC8ifSx7IngtYW16LWFsZ29yaXRobSI6IkFXUzQtSE1BQy1TSEEyNTYifSx7IngtYW16LWRhdGUiOiIyMDE5MDIxMlQxNjA0MzJaIn0seyJ4LWFtei1zZWN1cml0eS10b2tlbiI6IkZRb0daWEl2WVhkekVHa2FERS9kN1ZqY09aQWU2UXBrdVNMdkFWeUxPUkJaditZSHhGUGZUZi85THVBMnQ4NlZYYlhYeGhDV2QxZTgzbGNVSnhmSGQ4bjNkRmlzc291dGpIWG5pcGxwcW5rR2l2bXdFK0w1NkJMQ1JCcDRzYnBXVkVVNVFZcEQ0MkE1VC81Nlgvc2Z5UU5WRzFwMlY5VDBpMTZqWS9ybVpWZ2FLeXU2a0Job2duRGlia0dxb2EvK3FDYmZGcklTVEVJNGdYOWtNTC8wS2xZTFZ0endsbTN0MFpMZlV6Wit1cHlBMGhBaEh3aVlkcHk0TWFkTEU0N245Qi91VHpPMUo0cVdiY1dLaXMrbkMvTy96SlhuTnVhZ2JnV3FoK1NuZjJRM2hyaGlWQTQySVVxZG0xZmtoMVhsdHRCOEZrMDU3dnlUaVFsT1FjY21zczh4cEUrbUNHR0FEMFRYS0tuVmkrTUYifV19 \
-F content-type=application/octet-stream \
-F success_action_status=200 \
-F success_action_redirect= \
-F x-amz-signature=2583b27e19fdb5ff23950d6866dc3765c1f1bcd9c9b43d2fb2cbeb252e9e6507 \
-F x-amz-credential=ASIATGVJZKM3AKEZODGH/20190212/us-east-1/s3/aws4_request/ \
-F x-amz-algorithm=AWS4-HMAC-SHA256 \
-F x-amz-date=20190212T160432Z \
-F x-amz-server-side-encryption=AES256 \
-F x-amz-security-token=FQoGZXIvYXdzEGkaDE/d7VjcOZAe6QpkuSLvAVyLORBZv+YHxFPfTf/9LuA2t86VXbXXxhCWd1e83lcUJxfHd8n3dFissoutjHXniplpqnkGivmwE+L56BLCRBp4sbpWVEU5QYpD42A5T/56X/sfyQNVG1p2V9T0i16jY/rmZVgaKyu6kBhognDibkGqoa/+qCbfFrISTEI4gX9kML/0KlYLVtzwlm3t0ZLfUzZ+upyA0hAhHwiYdpy4MadLE47n9B/uTzO1J4qWbcWKis+nC/O/zJXnNuagbgWqh+Snf2Q3hrhiVA42IUqdm1fkh1XlttB8Fk057vyTiQlOQccmss8xpE+mCGGAD0TXKKnVi+MF \
-F file=#/C:\Users\sejjilali\Desktop\InventorForgeAddInServer.zip
From your postman request, it look like you have a space at the end of your x-amz-algorithm form-data value. Try removing it after "AWS4-HMAC-SHA256".
You are using Policy to specify an authorization token, the token needs to be a Header both in your Postman and cURL command "-H 'Authorization: Bearer [token]".
Use -H 'Content-Type: application/x-www-form-urlencoded' instead of the octet-stream.

Extract token from a curl request and use in another shell command

I've started picking up bash scripting and I'm stuck at something I'm trying to wrap my head around.
I have a curl command that outputs a token and I need to use it in the following command:
curl -k 'https://server:port/session' -X POST -H 'Content-Type: application/json' -d '{"username":"admin","password":"password"}'
It then outputs a token here:
{"token":"ac07098ad59ca6f3fccea0e2a2f6cb080df55c9a52fc9d65"}
I then need to use it in the follow up command
curl https://server:port/ -k -X POST -H 'Content-Type: application/json' -H 'X-Cookie:token=token' -d '
I was thinking I could output the token to a file, then have a sed command write the token to a file, then the new command use a variable where token=$token
Thanks!
This is where a JSON parsing tool comes in handy (such as jq):
$ echo '{"token":"ac07098ad59ca6f3fccea0e2a2f6cb080df55c9a52fc9d65"}' | jq -r .token
ac07098ad59ca6f3fccea0e2a2f6cb080df55c9a52fc9d65
So
json=$( curl -k 'https://server:port/session' -X POST -H 'Content-Type: application/json' -d '{"username":"admin","password":"password"}' )
token=$( jq -r ".token" <<<"$json" )
curl https://server:port/ -k -X POST -H "X-Cookie:token=$token" ...
With no further tool than a bash (tested Centos/Rhel7/GitBash) :
json=$(curl -k 'https://server:port/session' \
-X POST -H 'Content-Type: application/json' \
-d '{"username":"admin","password":"password"}') \
&& token=$(echo $json | sed "s/{.*\"token\":\"\([^\"]*\).*}/\1/g") \
&& echo "token = $token"
then use your authentication needing commands like that :
curl https://server:port/ -k -X POST \
-H 'Content-Type: application/json' \
-H 'X-Cookie:token=$token' -d ...'
If Python is installed, and hopefully it is on modern systems, you can do something like:
OUTPUT="$(curl -k 'https://server:port/session' -X POST -H 'Content-Type: application/json' -d '{"username":"admin","password":"password"}' | python -c "import sys, json; print json.load(sys.stdin)['token']")"
This will give you:
echo $OUTPUT
ec2e99a1d294fd4bc0a04da852ecbdeed3b55671c08cc09f
Use the `` syntax:
cmd1result=$(command1 | cut -d ':' -f 2 | grep -Po "[a-z0-9-]+")
command2 $cmd1result

How to pass payload via JSON file for curl?

I can successfully create a place via curl executing the following command:
$ curl -vX POST https://server/api/v1/places.json -d "
auth_token=B8dsbz4HExMskqUa6Qhn& \
place[name]=Fuelstation Central& \
place[city]=Grossbeeren& \
place[address]=Buschweg 1& \
place[latitude]=52.3601& \
place[longitude]=13.3332& \
place[washing]=true& \
place[founded_at_year]=2000& \
place[products][]=diesel& \
place[products][]=benzin \
"
The server returns HTTP/1.1 201 Created.
Now I want to store the payload in a JSON file which looks like this:
// testplace.json
{
"auth_token" : "B8dsbz4HExMskqUa6Qhn",
"name" : "Fuelstation Central",
"city" : "Grossbeeren",
"address" : "Buschweg 1",
"latitude" : 52.3601,
"longitude" : 13.3332,
"washing" : true,
"founded_at_year" : 2000,
"products" : ["diesel","benzin"]
}
So I modify the command to be executed like this:
$ curl -vX POST http://server/api/v1/places.json -d #testplace.json
This fails returning HTTP/1.1 401 Unauthorized. Why?
curl sends POST requests with the default content type of application/x-www-form-urlencoded. If you want to send a JSON request, you will have to specify the correct content type header:
$ curl -vX POST http://server/api/v1/places.json -d #testplace.json \
--header "Content-Type: application/json"
But that will only work if the server accepts json input. The .json at the end of the url may only indicate that the output is json, it doesn't necessarily mean that it also will handle json input. The API documentation should give you a hint on whether it does or not.
The reason you get a 401 and not some other error is probably because the server can't extract the auth_token from your request.
To clarify how to actually specify a file that contains the JSON to post, note that it's with the # sign as shown in the OP
e.g. a typical post to a local .NET Core API:
curl -X POST https://localhost:5001/api -H "Content-Type: application/json" -d #/some/directory/some.json
You can cat the contents of a json file to curl via the --data-raw parameter
curl https://api.com/route -H 'Content-Type: application/json' --data-raw "$(cat ~/.json/payload-2022-03-03.json | grep -v '^\s*//')"
curl https://api.com/route -H 'Content-Type: application/json' -d #<(jq . ~/.json/payload-2022-03-03.json)
curl https://api.com/route -H 'Content-Type: application/json' -d #<(jq '{"payload": .}' < ~/.json/payload-2022-03-03.json)
Note: comments in the json file are filtered out via grep -v '^\s*//'
You can also pass the data to curl via stdin using grep or cat or jq
grep -v '^\s*//' ~/.json/payload-2022-03-03.json | curl https://api.com/route -H 'Content-Type: application/json' -d #-
cat ~/.json/payload-2022-03-03.json | grep -v '^\s*//' | curl https://api.com/route -H 'Content-Type: application/json' -d #-
jq . ~/.json/payload-2022-03-03.json | curl https://api.com/route -H 'Content-Type: application/json' -d #-
jq '{"payload": .}' < ~/.json/payload-2022-03-03.json | curl https://api.com/route -H 'Content-Type: application/json' -d #-