I was wondering if Wirecloud offers complete support for object storage with FI-WARE Testbed instead of Fi-lab. I have successfully integrated Wirecloud with Testbed and have developed a set of widgets that are able to upload/download files to specific containers in Fi-lab with success. However, the same widgets do not seem to work in Fi-lab, as i get an error 500 when trying to retrieve the auth tokens (also with the well known object-storage-test widget) containing the following response:
SyntaxError: Unexpected token
at Object.parse (native)
at create (/home/fiware/fi-ware-keystone-proxy/controllers/Token.js:343:25)
at callbacks (/home/fiware/fi-ware-keystone-proxy/node_modules/express/lib/router/index.js:164:37)
at param (/home/fiware/fi-ware-keystone-proxy/node_modules/express/lib/router/index.js:138:11)
at pass (/home/fiware/fi-ware-keystone-proxy/node_modules/express/lib/router/index.js:145:5)
at Router._dispatch (/home/fiware/fi-ware-keystone-proxy/node_modules/express/lib/router/index.js:173:5)
at Object.router (/home/fiware/fi-ware-keystone-proxy/node_modules/express/lib/router/index.js:33:10)
at next (/home/fiware/fi-ware-keystone-proxy/node_modules/express/node_modules/connect/lib/proto.js:195:15)
at Object.handle (/home/fiware/fi-ware-keystone-proxy/server.js:31:5)
at next (/home/fiware/fi-ware-keystone-proxy/node_modules/express/node_modules/connect/lib/proto.js:195:15)
I noticed that the token provided in the beggining (to start the transaction) is
token: Object
id: "%fiware_token%"
Any idea regarding what might have gone wrong?
The WireCloud instance available at FI-WARE's testbed is always the latest stable version while the FI-LAB instance is currently outdated, we're working on updating it as soon as possible. One of the things that changes between those versions is the Object Storage API, so sorry for the inconvenience as you will not be able to use widgets/operators using the Object Storage in both environments.
Anyway, the response you were obtaining seems to indicate the object storage instance you are accessing is not working properly, so you will need to send an email to one of the available mail lists for getting help (fiware-testbed-help or fiware-lab-help) telling what is happening to you (remember to include your account information as there are several object storage nodes and ones can be up and the others down).
Regarding the strange request body:
"token": {
id: "%fiware_token%"
}
This behaviour is normal, as the WireCloud client code has no direct access to the IdM token of the user. It's the WireCloud's proxy which replaces the %fiware_token% pattern with the correct value.
Related
I'm using pulumi azure native for infrastructure as code. I need to create an Azure Web App (based on an App Service Plan) and add some app settings (and connection strings) throughout the code, e.g., Application Insights instrumentation key, Blob Storage account name, etc.
I figured out that there is a method, WebAppApplicationSettings, that can update web app settings:
from pulumi_azure_native import web
web_app = web.WebApp(
'my-web-app-test123',
...
)
web.WebAppApplicationSettings(
'myappsetting',
name=web_app.name,
resource_group='my-resource-group',
properties={'mySetting': 123456},
opts=ResourceOptions(depends_on=[web_app])
)
It turns out that WebAppApplicationSettings replaces the entire app settings with the value given in the properties parameter, which is not what I need. I need to append a new setting to the existing settings.
So, I tried this:
Fetch the existing settings from web app using list_web_app_application_settings_output
Add the new settings the existing settings
Update the app settings using WebAppApplicationSettings
from pulumi_azure_native import web
app = web.WebApp(
'my-web-app-test123',
...
)
current_apps_settings = web.list_web_app_application_settings_output(
name=web_app.name,
resource_group_name='my-resource-group',
opts=ResourceOptions(depends_on=[web_app])
).properties
my_new_setting = {'mySetting': 123456}
new_app_settings = Output.all(current=current_apps_settings).apply(
lambda args: my_new_setting.update(args['current'])
)
web.WebAppApplicationSettings(
'myappsetting',
name=app.name,
resource_group='my-resource-group',
properties=new_app_settings,
opts=ResourceOptions(depends_on=[web_app])
)
However, this doesn't work either and throws the following error during pulumi up:
Exception: invoke of azure-native:web:listWebAppApplicationSettings failed: invocation of azure-native:web:listWebAppApplicationSettings returned an error: request failed /subscriptions/--------------/reso
urceGroups/pulumi-temp2/providers/Microsoft.Web/sites/my-web-app-test123/config/appsettings/list: autorest/azure: Service returned an error. Status=404 Code="ResourceNotFound" Message="The Resource 'Microsoft.Web/sites/my-web-app-test123' under resource group 'pulumi-temp2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"
error: an unhandled error occurred: Program exited with non-zero exit code: 1
Is there way that I can add a new app setting to Azure Web App using pulumi without changing/removing the existing settings?
Here's a suboptimal workaround: App Configuration and Enable Azure Function Dynamic Configuration.
And as far as I can tell it comes with some drawbacks:
cold start time may increase
additional costs
care must be taken to avoid redundant calls (costly)
additional boilerplate code needed for every function app
Maybe there's a better way, I mean I hope there is, I just haven't found it yet either.
After some searching and reaching out to pulumi-azure-native people, I found an answer:
Azure REST API doesn't currently support this feature, i.e., updating a single Web App setting apart from the others. So, there isn't such a feature in pulumi-azure-native as well.
As a workaround, I stored (kept) all the app settings I needed to be added, updated, or removed in a dictionary throughout my Python script, and then I passed them to the web.WebAppApplicationSettings class at the end of the script so that they will be applied all at once to the Web App resource. This is how I solved my problem.
I was able to connect and upload videos using the library but when I deleted the app connection on Vimeo.com (as a test) the app didn't authorize again.
the upload looks like it's working but nothing is uploaded as the app is no longer connected.
I deleted the app on the phone and restarted but it still won't re-authorize the app.
This comes up in the output:
Vimeo upload state : Executing
Vimeo upload state : Finished
Invalid http status code for download task.
And this is in OldVimeoUpload.swift: ( didn't include the actual access code!)
import Foundation
class OldVimeoUpload: VimeoUpload
{
static var VIMEO_ACCESS_TOKEN :String! // = "there's a string of numbers here"
static let sharedInstance = OldVimeoUpload(backgroundSessionIdentifier: "") { () -> String? in
return VIMEO_ACCESS_TOKEN // See README for details on how to obtain and OAuth token
}
// MARK: - Initialization
override init(backgroundSessionIdentifier: String, authTokenBlock: AuthTokenBlock)
{
super.init(backgroundSessionIdentifier: backgroundSessionIdentifier, authTokenBlock: authTokenBlock)
}
}
It looks like the access token number is commented out. I deleted the 2 forward slashes to see if that would fix it but it didn't.
I spoke too soon.
It sounds like you went to developer.vimeo.com and created an auth token. Used it to upload videos. And then went back to developer.vimeo.com and deleted the auth token.
The app / VimeoUpload will not automatically re-authenticated in this situation. You've killed the token and the app cannot request a new one for you. You'll need to create a new auth token and plug it into the app.
If this is not accurate and you're describing a different issue let us know.
If you inspect the error that's thrown from the failing request I'm guessing you'll see it's a 401 unauthorized related to using an invalid token.
Edit:
Disconnecting your app (as described in your comment below) has the same effect as deleting your auth token from developer.vimeo.com.
Also, VimeoUpload accepts a hardcoded auth token (as you see from the README and your code sample). It will not automatically re-authenticate, probably ever.
If you'd like to handle authentication in your app check out VimeoNetworking or VIMNetworking. Either of those libraries can be used to create a variety of authentication flows / scenarios. Still, if a logged in user disconnects or deletes their token, you will need them to deliberately re-authenticate (i.e. you will need to build that flow yourself). In that case, the user has explicitly stated that they don't want the app to be able to access information on their behalf. It would go against our security contract with them to automatically re-authenticate somehow.
Does that make sense?
im trying to connect my polymer element to my own elasticsearch-server.
My first problem was, that they are on two different ports, so it had to choose JSONP because of Cross-Domain problems.
So I found out, that I just have to add
http.jsonp.enable: true
in the elasticsearch.yml.
Im starting the server simply by executing the "elasticsearch.bat".
I've indexed data.
If I try to load the API via iron-jsonp-library, im always getting an unexpected token error.
<iron-jsonp-library id="libraryLoader"
library-url="http://127.0.0.1:9200/data/_search?pretty%%callback%%"
notify-event="api-load"
callbackName="jsonpCallback">
</iron-jsonp-library>
In Google Chrome, I'm getting following result from elasticsearch
{"took":2,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":5,"max_score":1.0,"hits":[{"_index":"data","_type":"data","_id":"5","_score":1.0,"_source":{"id":5,"name":"Meyr","manufacturer":"Meyr","weight":1.0,"price":1.0000,"popularity":1,"instock":true,"includes":"Meyr"}},{"_index":"data","_type":"data","_id":"2","_score":1.0,"_source":{"id":2,"name":"Meier","manufacturer":"Meier","weight":1.0,"price":1.0000,"popularity":1,"instock":true,"includes":"Meier"}},{"_index":"data","_type":"data","_id":"4","_score":1.0,"_source":{"id":4,"name":"Mair","manufacturer":"Mair","weight":1.0,"price":1.0000,"popularity":1,"instock":true,"includes":"Mair"}},{"_index":"data","_type":"data","_id":"1","_score":1.0,"_source":{"id":1,"name":"Maier","manufacturer":"Maier","weight":1.0,"price":1.0000,"popularity":1,"instock":true,"includes":"Maier"}},{"_index":"data","_type":"data","_id":"3","_score":1.0,"_source":{"id":3,"name":"Mayr","manufacturer":"Mayr","weight":1.0,"price":1.0000,"popularity":1,"instock":true,"includes":"Mayr"}}]}}
Due to some internet knowledge of JSONP, its not jsonp.
Why is my elasticsearch server, not formatting right?
Are you prior to v2.0? Looks like they removed jsonp in 2.0 (elastic.co/guide/en/elasticsearch/reference/2.2/…).
Alsopretty%%callback%% doesn't look right, the %%callback%% macro usually needs to be the value of name (like onload=%%callback%%). The element replaces %%callback%% with the name of a global function that is generated for you.
I'm building an application that stores files into the FIWARE Object Storage. I don't quite understand what is the correct way of storing files into the storage.
The code python code snippet below taken from the Object Storage - User and Programmers Guide shows 2 ways of doing it:
def store_text(token, auth, container_name, object_name, object_text):
headers = {"X-Auth-Token": token}
# 1. version
#body = '{"mimetype":"text/plain", "metadata":{}, "value" : "' + object_text + '"}'
# 2. version
body = object_text
url = auth + "/" + container_name + "/" + object_name
return swift_request('PUT', url, headers, body)
The 1. version confuses me, because when I first looked at the only Node.js module (repo: fiware-object-storage) that works with Object Storage, it seemed to use 1. version. As the module was making calls to the old (v.1.1) API version instead of the presumably newest (v.2.0), referencing to the python example, not sure if that is an outdated version of doing it or not.
As I played more with the module, realised it didn't work and the code for it was a total mess. So I forked the project and quickly understood that I will need rewrite it form the ground up, taking the above mention python example from the usage guide as an reference. Link to my repo.
As of writing this the only methods that aren't implement is the object storage (PUT) and object fetching (GET).
Had some addition questions about the Object Storage which I sent to fiware-lab-help#lists.fiware.org, but haven't heard anything back so asking them here.
Haven't got much experience with writing API libraries. Should I need to worry about auth token expiring? I presume it is not needed to make a new authentication, every time we interact with storage. The authentication should happen once when server is starting-up (we create a instance) and it internally keeps it. Should I implement some kind of mechanism that refreshes the token?
Does the tenant id change? From the quote below is presume that getting a tenant I just a one time deal, then later you can use it in the config to make less authentication calls.
A valid token is required to access an object store. This section
describes how to get a valid token assuming an identity management
system compatible with OpenStack Keystone is being used. If the
username, password and tenant details are known, only step 3 is
required. source
During the authentication when fetching tenants how should I select the "right" one? For now i'm just taking the first one similar as the example code does.
Is it true that a object storage container belongs to only a single region?
Use only what you call version 2. Ignore your version 1. It is commented out in the example. It should be removed from the documentation.
(1) The token will be valid for some period of time. This could be an hour or a day, depending on the setup. This period of time should be specified in the token that is returned by the authentication service. The token needs to be periodically refreshed.
(2) The tenant id does not change.
(3) Typically only one tenant id is returned. It is possible, however, that you were assigned more than one id, in which case you have to pick which one you are currently using. Containers typically belong to a single tenant and are not shared between tenants.
(4) Containers are typically limited to a single region. This may change in the future when multi-region support for a container is added to Swift.
Solved my troubles and created the NPM module that works with the FIWARE Object Storage: https://github.com/renarsvilnis/fiware-object-storage-ge
I've successfully installed and run the Google Drive Quick Start application called DriveCommandLine. I've also adapted it a little to GET file info for one of the files in my Drive account.
What I would like to do now is save the credentials somehow and re-use them without the user having to visit a web page each time to get an authorization code. I have checked out this page with instructions to Retrieve and Use OAuth 2.0 credentials. In order to use the example class (MyClass), I have modified the line in DriveCommandLine where the Credential object is instantiated:
Credential credential = MyClass.getCredentials(code, "");
This results in the following exception being thrown:
java.lang.NullPointerException
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:187)
at com.google.api.client.json.jackson.JacksonFactory.createJsonParser(JacksonFactory.java:84)
at com.google.api.client.json.JsonFactory.fromInputStream(JsonFactory.java:247)
at com.google.api.client.googleapis.auth.oauth2.GoogleClientSecrets.load(GoogleClientSecrets.java:168)
at googledrive.MyClass.getFlow(MyClass.java:145)
at googledrive.MyClass.exchangeCode(MyClass.java:166)
at googledrive.MyClass.getCredentials(MyClass.java:239)
at googledrive.DriveCommandLine.<init>(DriveCommandLine.java:56)
at googledrive.DriveCommandLine.main(DriveCommandLine.java:115)
I've been looking at these APIs (Google Drive and OAuth) for 2 days now and have made very little progress. I'd really appreciate some help with the above error and the problem of getting persistent credentials in general.
This whole structure seems unnecessarily complicated to me. Anybody care to explain why I can't just create a simple Credential object by passing in my Google username and password?
Thanks,
Brian O Carroll, Dublin, Ireland
* Update *
Ok, I've just gotten around the above error and now I have a new one.
The way I got around the first problem was by modifying MyClass.getFlow(). Instead of creating a GoogleClientServices object from a json file, I have used a different version of GoogleAuthorizationCodeFlow.Builder that allows you to enter the client ID and client secret directly as Strings:
flow = new GoogleAuthorizationCodeFlow.Builder(httpTransport, jsonFactory, "<MY CLIENT ID>", "<MY CLIENT SECRET>", SCOPES).setAccessType("offline").setApprovalPrompt("force").build();
The problem I have now is that I get the following error when I try to use flow (GoogleAuthorizationCodeFlow object) to exchange the authorization code for the Credentials object:
An error occurred: com.google.api.client.auth.oauth2.TokenResponseException: 400 Bad Request
{
"error" : "invalid_scope"
}
googledrive.MyClass$CodeExchangeException
at googledrive.MyClass.exchangeCode(MyClass.java:185)
at googledrive.MyClass.getCredentials(MyClass.java:262)
at googledrive.DriveCommandLine.<init>(DriveCommandLine.java:56)
at googledrive.DriveCommandLine.main(DriveCommandLine.java:115)
Is there some other scope I should be using for this? I am currently using the array of scopes provided with MyClass:
private static final List<String> SCOPES = Arrays.asList(
"https://www.googleapis.com/auth/drive.file",
"https://www.googleapis.com/auth/userinfo.email",
"https://www.googleapis.com/auth/userinfo.profile");
Thanks!
I feel your pain. I'm two months in and still getting surprised.
Some of my learnings...
When you request user permissions, specify "offline=true". This will ("sometimes" sic) return a refreshtoken, which is as good as a password with restricted permissions. You can store this and reuse it at any time (until the user revokes it) to fetch an access token.
My feeling is that the Google SDKs are more of a hinderence than a help. One by one, I've stopped using them and now call the REST API directly.
On your last point, you can (just) use the Google clientlogin protocol to access the previous generation of APIs. However this is totally deprecated and will shortly be turned off. OAuth is designed to give fine grained control of authorisation which is intrinsically complex. So although I agree it's complicated, I don't think it's unnecessarily so. We live in a complicated world :-)
Your and mine experiences show that the development community is still in need of a consolidated document and recipes to get this stuff into our rear-view mirrors so we can focus on the task at hand.
Oath2Scopes is imported as follows:
import com.google.api.services.oauth2.Oauth2Scopes;
You need to have the jar file 'google-api-services-oauth2-v2-rev15-1.8.0-beta.jar' in your class path to access that package. It can be downloaded here.
No, I don't know how to get Credentials without having to visit the authorization URL at least once and copy the code. I've modified MyClass to store and retrieve credentials from a database (in my case, it's a simple table that contains userid, accesstoken and refreshtoken). This way I only have to get the authorization code once and once I get the access/refresh tokens, I can reuse them to make a GoogleCredential object. Here's how Imake the GoogleCredential object:
GoogleCredential credential = new GoogleCredential.Builder().setJsonFactory(jsonFactory)
.setTransport(httpTransport).setClientSecrets(clientid, clientsecret).build();
credential.setAccessToken(accessToken);
credential.setRefreshToken(refreshToken);
Just enter your clientid, clientsecret, accessToken and refreshToken above.
I don't really have a whole lot of time to separate and tidy up my entire code to post it up here but if you're still having problems, let me know and I'll see what I can do. Although, you are effectively asking a blind man for directions. My understanding of this whole system is very sketchy!
Cheers,
Brian
Ok, I've finally solved the second problem above and I'm finally getting a working GoogleCredential object with an access token and a refresh token.
I kept trying to solve the scopes problem by modifying the list of scopes in MyClass (the one that manages credentials). In the end I needed to adjust the scopes in my modified version of DriveCommandLine (the one that's originally used to get an authorization code). I added 2 scopes from Oauth2Scopes:
GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow.Builder(
httpTransport, jsonFactory, CLIENT_ID, CLIENT_SECRET,
Arrays.asList(DriveScopes.DRIVE, Oauth2Scopes.USERINFO_EMAIL, Oauth2Scopes.USERINFO_PROFILE))
.setAccessType("offline").setApprovalPrompt("force").build();
Adding the scopes for user information allowed me to get the userid later in MyClass. I can now use the userid to store the credentials in a database for re-use (without having to get the user to go to a URL each time). I also set the access type to "offline" as suggested by pinoyyid.