On our Stack users by default have discoverer permissions on resources. I was surprised that this also gives users the ability to query the last_transaction_rid of the dataset (using catalog/datasets/<rid>/reverse-transactions2/<branch>), so this method to check if a user has access is not working.
What would be the recommended and most performant API call to check if I, with my current Foundry token, can read the actual content of a dataset? Note: I don't want to query the content, but just understand if I would have the permissions to do so.
Would Data Lineage App (Monocle) work for you?
Open it workspace/data-integration/monocle/ > Find the dataset(s) > top right dropdown ("Resource Type" -> "Permissions")
The API that powers it can be extracted by checking the Dev Tools > Network Tab if you want / need to automate it (I've done this in the past and worked well for my use case)
I have resorted back to calling the compass/resources API and checking if response['operations'] contains compass:view:
def get_dataset_details(api_base: str, dataset_path_or_rid: str, headers: dict) -> dict:
"""
Returns the resource information of a dataset
Args:
dataset_path_or_rid: The full path or rid to the dataset
Returns: (dict) the json response of the api
"""
if 'ri.foundry.main.dataset' in dataset_path_or_rid:
response = requests.get(f"{api_base}/compass/api/resources/{dataset_path_or_rid}",
headers=headers,
params={'decoration': 'path'})
else:
response = requests.get(f"{api_base}/compass/api/resources",
headers=headers,
params={
'path': dataset_path_or_rid,
'decoration': 'path'
})
response.raise_for_status()
if response.status_code != 200:
raise ValueError(f"Dataset {dataset_path_or_rid} not found; "
f"If you are sure your dataset_path is correct, "
f"check if your jwt token "
f"is still valid!")
return response.json()
details = get_dataset_details(...)
if 'compass:view' not in dataset_details['operations']:
raise ValueError("No compass:view access to dataset")
Related
This question already has an answer here:
Gmail AppScript mailMessage.getFrom() to return email not name
(1 answer)
Closed 2 years ago.
I have developed a Gmail add-on.
In that I will get the list of emails in from, to, cc and bcc with the help of GmailMessage class.
like for from email , GmailMessage.getFrom()
for to email, GmailMessage.getTo()
In the documentation it is stated that it will return an email as string but it is not clearly mentioned anywhere about the format of that string.
Sometimes I'm getting in the format of Name of the account<accountEmail#gmail.com>
and sometimes in the format of <accountEmail#gmail.com> and sometimes only name will be available without the email ID of the recepient.
Sometimes it also includes double quotes on the email like Name of the account<"accountEmail#gmail.com">
Can someone tell is there a way to get the exact email address and name separately from Gmail using Google apps script?
Or suggest some ways to parse the email from GmailMessage class and get email address and name separately
I found the the gmail api site (here) which might come in use if you end up using the code. Although the code is in python so it might not work on google app script, but the only reason to use google apps script is to achieve these results. You also are going to have to know what you are doing when using this(if you want to use google app script, try making a gmail app template then go here to learn the basics (here) and here for further documentation (here))
Replace CLIENTSECRETS_LOCATION value with the location of your client_secrets.json file.
import logging
from oauth2client.client import flow_from_clientsecrets
from oauth2client.client import FlowExchangeError
from apiclient.discovery import build
# ...
# Path to client_secrets.json which should contain a JSON document such as:
# {
# "web": {
# "client_id": "[[YOUR_CLIENT_ID]]",
# "client_secret": "[[YOUR_CLIENT_SECRET]]",
# "redirect_uris": [],
# "auth_uri": "https://accounts.google.com/o/oauth2/auth",
# "token_uri": "https://accounts.google.com/o/oauth2/token"
# }
# }
CLIENTSECRETS_LOCATION = '<PATH/TO/CLIENT_SECRETS.JSON>'
REDIRECT_URI = '<YOUR_REGISTERED_REDIRECT_URI>'
SCOPES = [
'https://www.googleapis.com/auth/gmail.readonly',
'https://www.googleapis.com/auth/userinfo.email',
'https://www.googleapis.com/auth/userinfo.profile',
# Add other requested scopes.
]
class GetCredentialsException(Exception):
"""Error raised when an error occurred while retrieving credentials.
Attributes:
authorization_url: Authorization URL to redirect the user to in order to
request offline access.
"""
def __init__(self, authorization_url):
"""Construct a GetCredentialsException."""
self.authorization_url = authorization_url
class CodeExchangeException(GetCredentialsException):
"""Error raised when a code exchange has failed."""
class NoRefreshTokenException(GetCredentialsException):
"""Error raised when no refresh token has been found."""
class NoUserIdException(Exception):
"""Error raised when no user ID could be retrieved."""
def get_stored_credentials(user_id):
"""Retrieved stored credentials for the provided user ID.
Args:
user_id: User's ID.
Returns:
Stored oauth2client.client.OAuth2Credentials if found, None otherwise.
Raises:
NotImplemented: This function has not been implemented.
"""
# TODO: Implement this function to work with your database.
# To instantiate an OAuth2Credentials instance from a Json
# representation, use the oauth2client.client.Credentials.new_from_json
# class method.
raise NotImplementedError()
def store_credentials(user_id, credentials):
"""Store OAuth 2.0 credentials in the application's database.
This function stores the provided OAuth 2.0 credentials using the user ID as
key.
Args:
user_id: User's ID.
credentials: OAuth 2.0 credentials to store.
Raises:
NotImplemented: This function has not been implemented.
"""
# TODO: Implement this function to work with your database.
# To retrieve a Json representation of the credentials instance, call the
# credentials.to_json() method.
raise NotImplementedError()
def exchange_code(authorization_code):
"""Exchange an authorization code for OAuth 2.0 credentials.
Args:
authorization_code: Authorization code to exchange for OAuth 2.0
credentials.
Returns:
oauth2client.client.OAuth2Credentials instance.
Raises:
CodeExchangeException: an error occurred.
"""
flow = flow_from_clientsecrets(CLIENTSECRETS_LOCATION, ' '.join(SCOPES))
flow.redirect_uri = REDIRECT_URI
try:
credentials = flow.step2_exchange(authorization_code)
return credentials
except FlowExchangeError, error:
logging.error('An error occurred: %s', error)
raise CodeExchangeException(None)
def get_user_info(credentials):
"""Send a request to the UserInfo API to retrieve the user's information.
Args:
credentials: oauth2client.client.OAuth2Credentials instance to authorize the
request.
Returns:
User information as a dict.
"""
user_info_service = build(
serviceName='oauth2', version='v2',
http=credentials.authorize(httplib2.Http()))
user_info = None
try:
user_info = user_info_service.userinfo().get().execute()
except errors.HttpError, e:
logging.error('An error occurred: %s', e)
if user_info and user_info.get('id'):
return user_info
else:
raise NoUserIdException()
def get_authorization_url(email_address, state):
"""Retrieve the authorization URL.
Args:
email_address: User's e-mail address.
state: State for the authorization URL.
Returns:
Authorization URL to redirect the user to.
"""
flow = flow_from_clientsecrets(CLIENTSECRETS_LOCATION, ' '.join(SCOPES))
flow.params['access_type'] = 'offline'
flow.params['approval_prompt'] = 'force'
flow.params['user_id'] = email_address
flow.params['state'] = state
return flow.step1_get_authorize_url(REDIRECT_URI)
def get_credentials(authorization_code, state):
"""Retrieve credentials using the provided authorization code.
This function exchanges the authorization code for an access token and queries
the UserInfo API to retrieve the user's e-mail address.
If a refresh token has been retrieved along with an access token, it is stored
in the application database using the user's e-mail address as key.
If no refresh token has been retrieved, the function checks in the application
database for one and returns it if found or raises a NoRefreshTokenException
with the authorization URL to redirect the user to.
Args:
authorization_code: Authorization code to use to retrieve an access token.
state: State to set to the authorization URL in case of error.
Returns:
oauth2client.client.OAuth2Credentials instance containing an access and
refresh token.
Raises:
CodeExchangeError: Could not exchange the authorization code.
NoRefreshTokenException: No refresh token could be retrieved from the
available sources.
"""
email_address = ''
try:
credentials = exchange_code(authorization_code)
user_info = get_user_info(credentials)
email_address = user_info.get('email')
user_id = user_info.get('id')
if credentials.refresh_token is not None:
store_credentials(user_id, credentials)
return credentials
else:
credentials = get_stored_credentials(user_id)
if credentials and credentials.refresh_token is not None:
return credentials
except CodeExchangeException, error:
logging.error('An error occurred during code exchange.')
# Drive apps should try to retrieve the user and credentials for the current
# session.
# If none is available, redirect the user to the authorization URL.
error.authorization_url = get_authorization_url(email_address, state)
raise error
except NoUserIdException:
logging.error('No user ID could be retrieved.')
# No refresh token has been retrieved.
authorization_url = get_authorization_url(email_address, state)
raise NoRefreshTokenException(authorization_url)
First timer when it comes to connecting to API. I'm trying to pull data from Toggl using my API token but I can't get credentials working. I tried to replicate the method by Chris Webb (https://blog.crossjoin.co.uk/2014/03/26/working-with-web-services-in-power-query/) but I can't get it working. Here's my M code:
let
Source = Web.Contents(
"https://toggl.com/reports/api/v2/details?workspace_id=xxxxx&client=xxxxxx6&billable=yes&user_agent=xxxxxxx",
[
Query=[ #"filter"="", #"orderBy"=""],
ApiKeyName="api-token"
])
in
Source
After that I'm inputting my API Token into Web API method in Access Web content windows but I get an error that credentials could not be authenticated. Here's Toggl API specification:
https://github.com/toggl/toggl_api_docs/blob/master/reports.md
Web.Contents function receives two parameters: url + options
Inside options, you define the headers and the api_key, and other queryable properties, such as:
let
baseUrl = "https://toggl.com/",
// the token part can vary depending on the requisites of the API
accessToken = "Bearer" & "insert api token here"
options = [
Headers = [Authorization = accessToken, #"Content-Type" =
"application/Json"], RelativePath ="reports/api/v2/details", Query =
[workspace_id=xxxxx, client=xxxxxx6 , billable=yes, user_agent=xxxxxxx]
]
Source = Web.Contents(baseUrl, options)
// since Web.Contents() doesn't parse the binaries it fetches, you must use another
// function to see if the data was retreived, based on the datatype of the data
parsedData = Json.Document(Source)
in
parsedData
The baseUrl is the smallest url that works and never changes;
The RelativePath is the next part of the url before the first "?".
The Query record is where you define all the attributes to query as a record.
This is usually the format, but check the documentation of the API you're querying to see if it is similar.
I recently designed a REST API using flask for a sample project. The front end was based on React.JS. But i got a feedback from a colleague that the API is not REST API and its RPC.
The API basically accepts 3 parameters, 2 numbers and a operation ('add','sub','mul','div'). on an end point http://127.0.0.1:5000/calculator
The input JSON will look like:
{"value1":"7.1","value2":"8","operator":"mul"}
from flask import Flask, jsonify, request, abort
from flask_cors import CORS
APP = Flask(__name__, static_url_path='')
CORS(APP) # For cross origin resource sharing
APP.config['CORS_HEADERS'] = 'Content-Type'
#APP.route('/calculator', methods=['POST'])
def calculator_operation():
if not request.json:
abort(400)
try:
val1 = float(request.json['value1'])
val2 = float(request.json['value2'])
operator = request.json['operator']
if operator == 'add':
result = val1 + vla2
elif operator == 'mul':
result = val1 * val2
elif operator == 'sub':
result = val1 - val2
elif operator == 'div' and val2 == 0:
result = 'Cant divide by 0'
elif operator == 'div':
result = round((val1 / val2), 2)
return (jsonify({'result': result}), 200)
except KeyError:
abort(400)
if __name__ == '__main__':
APP.run(debug=True)
The code works fine. I would like to know if this is REST or RPC based on the end points and the operation being performed.
EDIT:
Ajax Call
$.ajax({
type: "POST",
url: "http://127.0.0.1:5000/calculator",
data: JSON.stringify({
value1: arg1,
value2: arg2,
operator: this.state.operation
}),
contentType: "application/json",
dataType: "json",
success:( data ) => {
this.setState({ result: data.result, argumentStr: data.result });
},
error: (err) => {
console.log(err);
}
});
I would like to know if this is REST or RPC based on the end points and the operation being performed.
How does the client discover what the endpoint is, and what the input json looks like?
On the web, there would be a standard media type that describes forms; the representation of the form would include keys and values, a target URI, and an HTTP method to use. The processing rules would describe how to take the details of the form, and the values provided by the consumer, and from them construct an HTTP request.
That's REST: doing what we do on the web.
Another REST approach would be to define a link relation, perhaps "http://example.org/calculator", and a media type application/prs.calculator+json, and then document that in your context the "http://example.org/calculator" link relation indicates that the target URI responds to POST messages with payload application/prs.calculator+json. This is essentially what Atom Syndication and Atom Pub.
That's also REST.
Fielding made an interesting comment about an API he had designed
I should also note that the above is not yet fully RESTful, at least how I use the term. All I have done is described the service interfaces, which is no more than any RPC. In order to make it RESTful, I would need to add hypertext to introduce and define the service, describe how to perform the mapping using forms and/or link templates, and provide code to combine the visualizations in useful ways.
That said, if you are performing GET-with-a-payload, a semantically safe request with a body, then you are probably trapped in RPC thinking. Notice that on the web, parameterized reads are done by communicating to the client how to modify the target-uri (for instance, by appending a query string with data encoded according to standardized processing rules).
REST stands for REpresentational State Transfer. Your operation is stateless, therefore there is no state to transfer. Your operation does, however, accept arguments and return a result in the manner of a procedure or function, and it is remote, so Remote Procedure Call would be a good description of what's going on. You are, after all, providing a language-independent way for clients to call your calculator_operation procedure.
What's conspicuously missing from your model is server-side data. In general, a REST API provides a way to interact with server-side objects: to query, create, replace, update, or delete. There's no data kept on your server side: it's just an oracle which answers questions. You have the "query" aspect and nothing else.
I have written a service which takes a json as input. I am using the website hurl.it to send post requests to check. Below is my code snippet:
class BatchSemanticSimilarityHandler(tornado.web.RequestHandler):
def post(self):
self.set_header('Access-Control-Allow-Origin', '*')
self.set_header('Access-Control-Allow-Credentials', 'true')
self.set_header('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, OPTIONS')
self.set_header('Access-Control-Allow-Headers','Origin, Accept, Content-Type, X-Requested-With, X-CSRF-Token')
data = json.loads(self.request.body)
apikey = data["apikey"]
try:
UA = self.request.headers["User-Agent"]
except:
UA = "NA"
if bool(usercoll.find_one({"apikey":apikey})) == True:
sentence = data["sentence"]
sentence_array = data["sentence_array"]
n = data["num_of_results"]
if sentence is None or sentence_array is [] or apikey is None or n is None:
self.set_status(200)
output = {"error":[{"code":334,"message":"Bad Input data"}]}
misscoll.insert({"apitype":"batchsemanticsimilarity","timestamp":datetime.datetime.now(), "ip":self.request.remote_ip, "useragent":UA, "uri":self.request.uri,"apikey":apikey, "output":output, "input":{"s1":sentence,"s2":sentence_array}})
self.write(output)
return
results = nb.get_similar(sentence, sentence_array, apikey, n)
print "results is",results
output = {"similar_sentences": results, 'credits':'ParallelDots'}
hitscoll.insert({"apitype":"batchsemanticsimilarity","timestamp":datetime.datetime.now(), "ip":self.request.remote_ip, "useragent":UA, "uri":self.request.uri,"apikey":apikey, "output":output, "input":{"s1":sentence,"s2":sentence_array}})
self.write(output)
return
else:
rejectcoll.insert({"apitype":"batchsemanticsimilarity","apikey":apikey,"timestamp":datetime.datetime.now(), "ip":self.request.remote_ip, "useragent":UA, "url":self.request.uri})
self.write({"error":[{"code":333,"message": "Bad Authentication data"}]})
return
The json that I am giving as the body of the request is as below:
{
"sentence": "BJP leads in Bengaluru civic body`s poll, all eyes on JD(S)",
"sentence_array": [
"Narendra Modi is the prime minister",
"Sonia Gandhi runs Congress",
"Sachin is a good batsman"
],
"apikey": "DyMe1gSNhvMV1I1b20a7KARYIwuQX5GAQ",
"num_of_results": 2
}
I have verified on jsonlint that this is a valid JSON.
However while sending the request it gives me below error:
ValueError: No JSON object could be decoded
Can anyone please help me sort this out!!
The JSON object that you are passing in POST request is encoded into the url.
JSON library cannot read the encoded data.So you need to decode the url first.
Decoding of url can be done using urlparse library in python.so you need something like this.
post_data=urlparse.parse_qsl(self.request.body)
According to your need of final format to read there are various methods in urlparse.check this
or
As specified in the docs you can override a method to enable JSON parsing
def prepare(self):
if self.request.headers["Content-Type"].startswith("application/json"):
self.json_args = json.loads(self.request.body)
else:
self.json_args = None
check this
I modified the https://developers.google.com/drive/v2/reference/files/list example to get a list of all file items, using request partial resources.
def retrieve_all_files(service, fields = None):
"""Retrieve a list of File resources.
Args:
service: Drive API service instance.
fields: The fields (of each files Resource item!), e.g. 'id,title'
if not passed, get everything.
Returns:
List of File resources.
"""
result = []
page_token = None
while True:
try:
param = {}
if fields:
param['fields'] = 'nextPageToken,items(' + fields + ')'
if page_token:
param['pageToken'] = page_token
files = service.files().list(**param).execute()
result.extend(files['items'])
page_token = files.get('nextPageToken')
if not page_token:
break
except errors.HttpError, error:
print 'An error occurred: %s' % error
break
return result
In my test Drive folder I have a file that has a description string set. When I run
retrieve_all_files(drive_service)
without setting the fields parameter (so get everything), sure enough in the list of items, the file looks fine:
[{u'alternateLink': u'...',...
u'description': u'Description',
u'id': u'...',...},...]
but running
retrieve_all_files(drive_service, fields='id,description,title')
yields a list where none of the items have the 'description' property. Not all files have the 'description' property set; is this why the partial resource response is omitting the field even from items that have the property? I'm not sure if this is an undocumented feature that I didn't expect or a bug.
I was able to reproduce your issue. When using an authorized scopes list of:
https://www.googleapis.com/auth/drive
which should be full access, I was not able to get the description field returned. However, I found that if I also included the scope:
https://www.googleapis.com/auth/drive.appdata
the description field would return properly.
Can you confirm on your end that adding the above to your scopes resolves the issue? Seems like a bug on Google's end.
This was confirmed as a bug and a fix should be available in a few days.