I'm using the drive change feed to find changes of files in Google Drive. I have been experiencing some inconsistency. I just tested using the API Explorer found here :
https://developers.google.com/drive/v2/reference/changes/list
On one user the largestChangeId returned was 1208, although when setting the startChangeId to 1208, the request returned 6 entries, including the change of 1208, and each other change being higher than 1208. I also have the flags includeDeleted = true, includeSubscribed = false.
I then tried on another user, the returned ID was 136023. When using this value with the same params as above Drive returned 0 entries, which I expected.
Then I tried on a third user, the returned ID was 8267. Then setting the call again with the same params as above Drive returned 1 entries, which had the same change id of 8267.
Note all three accounts are testing accounts on a test domain in which no documents are being modified or shared.
So three users three results making the same call. Is this expected or a better question, what should I expect when making this call. In the documentation it states : " Change ID to start listing changes from. (string)". When it says from and you are returned the highest ID I would expect 0 results to be returned.
Thanks.
Related
I've been working with the drive SDK for the last few days and previously I was able to request 1000 items on a page using listrequest. Now I'm only getting 100 files no matter how high I set the parameter, but if I set it below 100 it will return whatever number. Did they decide to change the max results overnight or something?
I'm facing the same problem so I've been doing some tests and this is what I've found:
If I try files.list with fields: "files, nextPageToken", I get 100 files only.
If I try files.list with fields: "files(name, id, etc, parents), nextPageToken", I get 460 files.
If I try files.list with fields: "files(name, id, etc), nextPageToken" without parents field, I get 1000 files.
If I try files.list with fields: "files(parents), nextPageToken", I get 460 files.
So it seems that depends on how many fields you request and also if one of the fields requested is parents field.
You've misunderstood what pagesize does. It is a maximum value, not an absolute value. You should always iterate list results until nextPageToken is null.
Ok never mind I figured this out. I was requesting all the metadata of each file, which seems to limit the request to 100. I was able to get 1000 per page by requesting only three fields.
EDIT by pinoyyid: This is not the answer. There is nothing that you can do to guarantee that the page will have exactly pageSize items.
I'm trying to implement the JSON api (v2) of bigquery. In my code I get the same behaviour as on the documentation page for tabledata-list
My table size is about 11.000 rows. In the documentation page I fill in the following parameters:
ProjectId = X
DatasetId = Y
TableId = Z
MaxResults = 10000 #I want to paginate my results
This returns 10.000 rows and a pageToken. So I do the same request and now I set the page token so that I get the next page of results.
And that returns the same 10.000 rows as before. I expected this to do pagination as described on this page:
All collection.list methods return paginated results under certain circumstances. The number of results per page is controlled by the maxResults property
A page is a subset of the total number of rows. If your results are more than one page of data, the result data will have a nextPageToken property. To retrieve the next page of results, make another list call and include the token value as a URL parameter named pageToken.
Where do I go wrong?
EDIT:
My colleague pointed out to me that on the other documentation pages the result contains a nextPageToken except the response contains a pageToken. The difference being that where pageToken refers to the current page, the nextPageToken refers to the next page.
However the documentation states it should return a nextPageToken (except when there is no more data). But len(table) > len(result)
On the same page it's mentioned that there is a difference for TableData.List() call
The bigquery.tabledata.list method, which is used to page through
table data, uses a row offset value or a page token.
So for TableData.List() you must use the row offset value to paginate, and in order to access previous pages you can use your hashes from your session. This is built because with large volume and big data, you cannot pre-cache the next set of data from your worker pool.
You can help improving the documentation, by using the link on top right of each page that says: Feedback on this document feel free to use that to reach out with improvements.
Also you can submit issues to https://code.google.com/p/google-bigquery/issues/list
Unfortunately, the field returned for TableData.List() that contains the logical "next page token" is literally named "pageToken", rather than "nextPageToken".
Other APIs, like Datasets.List(), return a field literally named "nextPageToken" which contains the logical "next page token".
It's a case of inconsistent naming, but hopefully this helps clear up some confusion.
I moved three files in My Drive to the trash. Then retrieved changes with Changes: list API (https://developers.google.com/drive/v2/reference/changes/list). It returned three changes, with ids 11607, 11608 and 11609. However, the largestChangeId field was 11608. When I made the API request again with startChangeId: 11608, it returned the two last changes. When I made the API request with startChangeId: 11609, it returned no result.
Is it expected? Or relying on change ids in such a way is not right?
In your case, it looks like you're exposed to a bug. I'm adding details of the changes list and how it should work in normal circumstances. Let's assume your latest change id was 11606 and you made three trashing operations.
GET changes?startChangeId=11607 should list:
11607
11608
11609
And the next time you are requesting changes.list, you should always increment the latest latestChangeId by 1.
In this case you need to request the following on your next poll.
GET changes?startChangeID=11610 and it will be returning an empty list.
Tinkering around with verifying a couple of domains and found the manual process rather tedius. My DNS controller offers API access so I figured why not script the whole thing.
Trick is I can't figure out how to access the required TXT & CNAME records for DKIMS verification from boto, when I punch in
dkims = conn.verify_domain_dkim('DOMAIN.COM')
it adds DOMAIN.COM to the list of domains pending verification but doesn't provide the needed records, the returned value of dkims is
{'VerifyDomainDkimResponse': {
'ResponseMetadata': {'RequestId': 'REQUEST_ID_STRING'},
'VerifyDomainDkimResult': {'DkimTokens': {
'member': 'DKIMS_TOKEN_STRING'}}}}
Is there some undocumented way to take the REQUEST_ID or TOKEN_STRING to pull up these records?
UPDATE
If you have an aws account you can see the records I'm after at
https://console.aws.amazon.com/ses/home?region=us-west-2#verified-senders:domain
tab: Details:: Record Type: TXT (Text)
tab: DKIM:: DNS Record 1, 2, 3
these are the records required to add to the DNS controller to validate & allow DKIM signatures to take place
This is how I do it with python.
DOMINIO = 'mydomain.com'
from boto3 import Session
session = Session(
aws_access_key_id=MY_AWS_ACCESS_KEY_ID,
aws_secret_access_key=MY_AWS_SECRET_ACCESS_KEY,
region_name=MY_AWS_REGION_NAME)
client = session.client('ses')
# gets VerificationToken for the domain, that will be used to add a TXT record to the DNS
result = client.verify_domain_identity(Domain=DOMINIO)
txt = result.get('VerificationToken')
# gets DKIM tokens that will be used to add 3 CNAME records
result = client.verify_domain_dkim(Domain=DOMINIO)
dkim_tokens = result.get('DkimTokens') # this is a list
At the end of the code, you will have "txt" and "dkim_tokens" variables, a string and a list respectively.
You will need to add a TXT record to your dns, where the host name is "_amazonses" and the value is the value of "txt" variable.
Also you will need to add 3 CNAME records to your dns, one for each token present in "dkim_tokens" list, where the host name of each record is of the form of [dkimtoken]._domainkey and the target is [dkimtoken].dkim.amazonses.com
After adding the dns records, after some minutes (maybe a couple of hours), Amazon will detect and verify the domain, and will send you an email notification. After that, you can enable Dkim signature by doing this call:
client.set_identity_dkim_enabled(Identity=DOMINIO, DkimEnabled=True)
The methods used here are verify_domain_identity, verify_domain_dkim and set_identity_dkim_enabled.
You may also want to take a look a get_identity_verification_attributes and get_identity_dkim_attributes.
I think the get_identity_dkim_attributes method will return the information you are looking for. You pass in the domain name(s) you are interested in and it returns the status for that identity as well as the DKIM tokens.
Using the Box 1.0 REST API, I am trying to work with the functions in SOAP UI.
The API doc for get_managed_users with user_id=12345 (internal id retrieved with get_user_id call correctly) is returning all the users. The docs say that would be the case if you do not specify a user_id value. But my full command is: (Token and API key changed to protect the clueless)
https://www.box.com/api/1.0/rest?user_id=27360&auth_token=blahbalhblah1234&action=get_managed_users&api_key=someKeyYouShouldNotSee
Now I could work with the complete result list, but that won't scale as we get thousands of users into the system.
I can make a call with edit_managed_user, using the same user_id value and the change is reflected in the UI, and in the next get_managed_users call. Thus I do have the correct user_id value, I would so assume.
I tried testuser#gmail.com as the user_id value as well, and get the entire list back. This leads me to believe that somehow I am sending user_id wrong, but I just do not see it.
Any hints? Why, with what seems like a valid user_id value is it acting like it is absent or incorrect?
Most likely you have either called this method with an invalid user_id, or one that is not in your set of managed users. Can you double check that the user comes back in your list of already managed users?