what is the best way to post http request from ssis package - ssis

i want to call a http post request through my ssis package. But not sure what should be the best way of calling a rest api post method. Please help .

You can use this method within a Script Task to make the api call using httpclient:
public void Post(Produto produto)
{
var client = new HttpClient;
client.BaseAddress = new Uri("https://localhost:5001/api/");
client.DefaultRequestHeaders.Clear();
//client.DefaultRequestHeaders.Accept.Clear();
//client.DefaultRequestHeaders.Add("ApiKey", "");
var jsonBody = JsonConvert.SerializeObject(produto);
client.PostAsync("createproduto", new StringContent(jsonBody, Encoding.UTF8, "application/json"));
}

You can make use of the namespace System.Net.WebClient to make the Http request with the help of Script Task in SSIS. Following example shows how this can be achieved. The example was created in SSIS 2008 R2.
public void Main()
{
Variables varCollection = null;
Dts.VariableDispenser.LockForRead("User::RemoteUri");
Dts.VariableDispenser.LockForRead("User::LocalFolder");
Dts.VariableDispenser.GetVariables(ref varCollection);
System.Net.WebClient myWebClient = new System.Net.WebClient();
string webResource = varCollection["User::RemoteUri"].Value.ToString();
string fileName = varCollection["User::LocalFolder"].Value.ToString() + webResource.Substring(webResource.LastIndexOf('/') + 1);
byte[] data;
using (WebClient client = new WebClient())
{
data = client.DownloadData(webResource);
}
FileInfo file = new System.IO.FileInfo(fileName);
file.Directory.Create(); // If the directory already exists, this method does nothing.
File.WriteAllBytes(file.FullName, data);
Dts.TaskResult = (int)ScriptResults.Success;
}

Related

Get Access Token - Design Automation for Revit

I have created the app, was able to debug the source code.
Also received Client ID and Client secret.
I need your help to understand how to get access token.
Basically this part -
enter image description here
It will be really helpful if you can provide a sample code on how to send the HTTP request as I am novice to web API's.
I have added this code to my solution -
enter image description here
enter image description here
Thanks,
STR
Here is a working example, how to obtain an Access token.
You need to add NewtonSoft.Json nuget package to your project to run it.
public class TokenModel
{
[JsonProperty("access_token")]
public string AccessToken;
}
public async Task<string> GetToken()
{
var credentials = new Dictionary<string, string>();
credentials.Add("client_id", "YOUR_CLIENT_ID");
credentials.Add("client_secret", "YOUR_CLIENT_SECRET");
credentials.Add("grant_type", "YOUR_GRANT_TYPE");
credentials.Add("scope", "YOUR_SCOPE");
var content = new FormUrlEncodedContent(credentials);
using (HttpClient client = new HttpClient())
{
client.BaseAddress = new Uri("https://developer.api.autodesk.com");
var response = await client.PostAsync("/authentication/v1/authenticate", content);
if (response.IsSuccessStatusCode)
{
var responseData = await response.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<TokenModel>(responseData).AccessToken;
}
}
}
And simple usage:
var token=await GetToken();
It is not perfect, I've tried to simplify all the moments.

Calling a webservice repeatedly and export data

I have a webservice which accepts a parameter and gives back result in a JSON message format.
http://portLocation/Company/WebService.asmx/Operation?parameter1=XX
My objective is to repeatedly call this webservice with the parameters in a CSV File.
What would be the ideal way to do it.
Here is a simple working concept for C#.
You'll need to adjust it for your needs.
static void Main(string[] args)
{
string datafile = "data.csv";
StringBuilder sbUrl = new StringBuilder("http://portLocation/Company/WebService.asmx/Operation");
List<string> fileContents = System.IO.File.ReadAllLines(datafile).ToList();
using (HttpClient httpClient = new HttpClient())
{
foreach (string item in fileContents)
{
string[] data = item.Split(',');
sbUrl.AppendFormat("?parameter1={0}", data[0]);
sbUrl.AppendFormat("&parameter2={0}", data[1]);
sbUrl.AppendFormat("&parameter3={0}", data[2]);
sbUrl.AppendFormat("&parameter4={0}", data[3]);
var response = httpClient.GetAsync(sbUrl.ToString());
}
}
}

View SharePoint 2010 list in JSON format

I am preparing to using Timeglider to create a timeline. One requirement is the data has to be in JSON format. One requirement for me is it needs to be client side as I do not have access to the servers or central admin.
When I try to do http://webname/_vti_bin/ListData.svc/listname I get an error for access permissions however when I issue it http://webname/subsite/_vti_bin/ListData.svc/listname I have no problem pulling data.
My situation is the list is on the TLD. I tried to follow this post How to retrieve a json object from a sharepoint list but it relates to SP 2007.
To implement pure JSON support in SharePoint 2007, 2010 and so on have a look at this project, http://camelotjson.codeplex.com/. It requires the commercial product Camelot .NET Connector to be installed on the server.
If you don't like to go commercial you can resort to the sp.js library, here is a small example I wrote, enjoy!
// Object to handle some list magic
var ListMagic = function () {
/* Private variables */
var that = this;
var clientContext = SP.ClientContext.get_current();
var web = clientContext.get_web();
var lists = web.get_lists();
/**
* Method to iterate all lists
*/
that.getLists = function () {
clientContext.load(lists);
clientContext.executeQueryAsync(execute, getFailed);
function execute() {
var listEnumerator = lists.getEnumerator();
while (listEnumerator.moveNext()) {
var l = listEnumerator.get_current();
// TODO! Replace console.log with actual routine
console.log(l.get_title());
}
}
function getFailed() {
// TODO! Implement fail management
console.log('Failed.');
}
};
/**
* Method to iterate all fields of a list
*/
that.getFields = function (listName) {
// Load list by listName, if not stated try to load the current list
var loadedList = typeof listName === 'undefined' ? lists.getById(SP.ListOperation.Selection.getSelectedList()) : that.lists.getByTitle(listName);
var fieldCollection = loadedList.get_fields();
clientContext.load(fieldCollection);
clientContext.executeQueryAsync(execute, getFailed);
function execute() {
var fields = fieldCollection.getEnumerator();
while (fields.moveNext()) {
var oField = fields.get_current();
// TODO! Replace console.log with actual routine
var listInfo = 'Field Title: ' + oField.get_title() + ', Field Name: ' + oField.get_internalName();
console.log(listInfo);
}
}
function getFailed() {
// TODO! Implement fail management
console.log('Failed.');
}
};
/**
* Method to get a specific listitem
*/
that.getListItem = function (itemId) {
var loadedList = lists.getById(SP.ListOperation.Selection.getSelectedList());
var spListItem = loadedList.getItemById(itemId);
clientContext.load(spListItem);
clientContext.executeQueryAsync(execute, getFailed);
function execute() {
// TODO! Replace console.log with actual routine
//spListItem.get_fieldValues()
console.log(spListItem.get_fieldValues()["Title"]);
}
function getFailed() {
// TODO! Implement fail management
console.log('Failed.');
}
};
/**
* Method to fake an init (optional)
*/
that.init = function () {
// Run any init functionality here
// I.e
that.getFields("Tasks");
};
return that;
};
// In case of no jquery use window.onload instead
$(document).ready(function () {
ExecuteOrDelayUntilScriptLoaded(function () {
var sp = new ListMagic();
sp.init();
}, 'sp.js');
});
Personally, I make HttpHandlers. I install them in the SharePoint isapi folder and the GAC and I can call them just like you might the owssvr.dll. http://servername/_vti_bin/myhttphandelr.dll
Pass it querystring variables or call it from jquery ajax. You can use the httpcontext and make a spcontext from it and have access to all sorts of information from the current location in SharePoint. Then you can javascriptserialize the objects and pass them as JSON. Looking for some code... Hang on... I can't put all the code but this should get you close. I use this to add a submenu to the context menu to allow a user to delete or rename a file if they uploaded it to a library and it is version 1.0 and to collect a file from a library and create a eml file with the selected file(s) as an attachment(s). We don't give our users delete privileges normally. Point being, you can now create a class with just the information you need from SharePoint and pass it as JSON. The only downfall I have with this, is iisreset is required if you make any changes to the dll.
I task schedule a iisreset every night at midnight anyway to keep it fresh and free from memory bloat. I come in the next day and my changes are there. The cool thing is, the spcontext has information about the current location in SharePoint from where it is called. So, http://servername/_vti_bin/myhttphandelr.dll vs http://servername/subsite/library/_vti_bin/myhttphandelr.dll
I might add. Don't try to serialize SharePoint objects. One they are huge, complex objects. Two, I don't think they are marked serializable. Just make you own class and populate it with the values you need from the SharePoint objects.
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices.ComTypes;
using System.Web;
using System.Web.Script.Serialization;
using ADODB;
using interop.cdosys;
using Microsoft.SharePoint;
namespace owssvr2
{
public class OWSsvr2 : IHttpHandler, System.Web.SessionState.IRequiresSessionState
{
private string cmd;
ctx ctx = new ctx();
private string currentuser;
private SPContext SPcontext;
private HttpContext cntx;
public bool IsReusable
{
get { return false; }
}
public void ProcessRequest(HttpContext context)
{
SPcontext = SPContext.GetContext(context); <-- Gets spcontext from the httpcontext
cntx = context;
ctx = GetData(context.Request); <-- I parse some information from the request to use in my app
cmd = ctx.Cmd;
ctx.User = context.User.Identity.Name;
currentuser = context.User.Identity.Name;
switch (cmd)
{
case "Delete":
Delete();
context.Response.Redirect(ctx.NextUsing);
break;
case "HasRights":
HasRights();
JavaScriptSerializer javaScriptSerializer = new JavaScriptSerializer();
string serEmployee = javaScriptSerializer.Serialize(ctx);
context.Response.Write(serEmployee);
context.Response.ContentType = "application/json; charset=utf-8";
break;
case "Rename":
Rename(context);
//context.Response.Redirect(context.Request["NextUsing"]);
break;
case "SendSingleFile":
try
{
context.Response.Clear();
context.Response.ClearHeaders();
context.Response.BufferOutput = true;
ADODB.Stream stream = SendSingleFile(context.Request["URL"]);
stream.Type = StreamTypeEnum.adTypeBinary;
stream.Position = 0;
context.Response.ContentType = "application/octet-stream";
context.Response.AddHeader("content-disposition", "attachment;filename=Email.eml");
IStream iStream = (IStream)stream;
byte[] byteArray = new byte[stream.Size];
IntPtr ptrCharsRead = IntPtr.Zero;
iStream.Read(byteArray, stream.Size, ptrCharsRead);
context.Response.BinaryWrite(byteArray);
context.Response.End();
}
catch(Exception ex) {context.Response.Write(ex.Message.ToString()); }
break;
case "SendMultiFile":
try
{
//SendMultiFile(context.Request["IDs"]);
context.Response.Clear();
context.Response.ClearHeaders();
context.Response.BufferOutput = true;
ADODB.Stream stream = SendMultiFile(context.Request["IDs"]);
stream.Type = StreamTypeEnum.adTypeBinary;
stream.Position = 0;
context.Response.ContentType = "application/octet-stream";
context.Response.AddHeader("content-disposition", "attachment;filename=Email.eml");
IStream iStream = (IStream)stream;
byte[] byteArray = new byte[stream.Size];
IntPtr ptrCharsRead = IntPtr.Zero;
iStream.Read(byteArray, stream.Size, ptrCharsRead);
context.Response.BinaryWrite(byteArray);
context.Response.End();
}
catch(Exception ex) {context.Response.Write("There was an error getting the files. </br>" + ex.Message.ToString()); }
break;
case "FileInfo":
JavaScriptSerializer javaScriptSerializer1 = new JavaScriptSerializer();
string serEmployee1 = javaScriptSerializer1.Serialize(FileInfo(context));
context.Response.Write(serEmployee1);
context.Response.ContentType = "application/json; charset=utf-8";
break;
case "UsersInGroups":
UsersInGroups ug = new UsersInGroups(context, context.Request["job"],context.Request["groups"]);
break;
}
}

Connecting SSIS WebService task to Spring WevService

I have a SSIS package in which i use a WebService task to call a Spring WS.
The authentication is done by client certificate and username & password.
I have tried to do it like this a simple HttpConnection and a WebService task - Error 504 Gateway Timeout. When i edit the HttpConnection and click on Test Connection i get an error that states:
"The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel."
I have tried doing it with a script task and the same error.
I have even tried with a dummy console application and the same result.
I also have a java written app that actually does the job but i do not have access to it's code-behind. This basically proves that the problem is not from the server itself.
The java application has it's own keystore and the same certificates that i have installed on the server.
I opened a wireshark capture and i saw that when i used either of my apps the host made a DNS request for an address that i did not configure anywhere(it seems like a proxy address from the intranet), while the java app made a DNS request with the correct address.
I am stuck here, and i have no idea what the problem might be or what else i can do so that i would get a proper error.
Please advise!
Edit:
This is the code that calls the WS:
public static void CallWebService()
{
var _url = "https://<IP>/App/soap/DataService";
string action = "getData";
Dictionary<string, string> parameters = new Dictionary<string, string>();
parameters.Add("param1", "0");
parameters.Add("param2", "0");
parameters.Add("param3", "value");
XmlDocument soapEnvelopeXml = CreateSoapEnvelope(action, parameters);
HttpWebRequest webRequest = CreateWebRequest(_url);
InsertSoapEnvelopeIntoWebRequest(soapEnvelopeXml, webRequest);
// begin async call to web request.
IAsyncResult asyncResult = webRequest.BeginGetResponse(null, null);
// suspend this thread until call is complete. You might want to
// do something usefull here like update your UI.
asyncResult.AsyncWaitHandle.WaitOne();
// get the response from the completed web request.
string soapResult;
using (WebResponse webResponse = webRequest.EndGetResponse(asyncResult))
{
using (StreamReader rd = new StreamReader(webResponse.GetResponseStream()))
{
soapResult = rd.ReadToEnd();
}
}
Console.WriteLine(soapResult);
}
private static HttpWebRequest CreateWebRequest(string url)
{
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.ContentType = "text/xml;charset=\"utf-8\"";
webRequest.Accept = "text/xml";
webRequest.Method = "POST";
string thumbprint = "CERTIFICATE THUMBPRINT";
byte[] thumbprintArray = new byte[thumbprint.Split(new char[]{ ' ' }).Length];
string[] stringArray = thumbprint.Split(new char[] { ' ' });
for (int i = 0; i < thumbprintArray.Length; i++)
{
thumbprintArray[i] = Convert.ToByte(stringArray[i], 16);
}
X509Store localStore = new X509Store("My");
localStore.Open(OpenFlags.ReadOnly);
X509Certificate2Collection certCol = localStore.Certificates.Find(X509FindType.FindByTimeValid, DateTime.Now, true);
foreach (X509Certificate cert in certCol)
{
if (cert.GetCertHashString() == thumbprint)
{
webRequest.ClientCertificates.Add(cert);
break;
}
}
webRequest.UseDefaultCredentials = false;
webRequest.Credentials = new NetworkCredential("USER", "PASSWORD");
return webRequest;
}
private static XmlDocument CreateSoapEnvelope(string action, Dictionary<string, string> parameters)
{
string formatedParameters = string.Empty;
string paramFormat = "<{0}>{1}</{0}>";
foreach (string key in parameters.Keys)
{
formatedParameters += string.Format(paramFormat, key, parameters[key]);
}
XmlDocument soapEnvelop = new XmlDocument();
soapEnvelop.LoadXml(string.Format(#"
<soapenv:Envelope xmlns:soap=""http://custom/soap/"" xmlns:soapenv=""http://schemas.xmlsoap.org/soap/envelope/"">
<soapenv:Header/>
<soapenv:Body>
<soap:{0}>
{1}
</soap:{0}>
</soapenv:Body>
</soapenv:Envelope>", action, formatedParameters));
return soapEnvelop;
}
private static void InsertSoapEnvelopeIntoWebRequest(XmlDocument soapEnvelopeXml, HttpWebRequest webRequest)
{
using (Stream stream = webRequest.GetRequestStream())
{
soapEnvelopeXml.Save(stream);
}
}

Webservices object reference not set to an instance of an object error

I have to make a call to the web service (JSON) to authenticate the user who is trying to login to the app. I have the following xml provided
<summary>
http://geniewebsvc.cloudapp.net/Member.svc/Authenticate
</summary>
<param name="payload">
{"UserName":"testuser#somedomain.com","Password":"p#$$w0rd"}
</param>
<requiredHeaders>
Content-Type: application/json;charset=UTF-8
</requiredHeaders>
<returns></returns>
[OperationContract]
[WebInvoke(UriTemplate = "/Authenticate", Method = "POST", ResponseFormat = WebMessageFormat.Json, RequestFormat = WebMessageFormat.Json, BodyStyle = WebMessageBodyStyle.Bare)]
AuthenticateResponse Authenticate(AuthCredentials usernamePassword);
There is similar one to check if the userid is already registered and that is a Get method. That works fine and i receive the right response for both successful and unsuccessful cases. But all the post methods are the ones which are giving me trouble. and i noticed that there is one more difference in these xmls.. i.e., the .. the isregistered webservice param tag goes something like this..
<param name="emailAddress"></param>
and here is my get() and post() please let me know whats my mistake...
public void Post()
{
RequestState myRequestState = new RequestState();
try
{
System.Uri uri = new Uri(url);
HttpWebRequest myHttpWebGetRequest;
Logger.log(TAG, "Create a HttpWebrequest object to the URL", url);
myHttpWebGetRequest = (HttpWebRequest)WebRequest.Create(uri);
_mHttpWebRequest = myHttpWebGetRequest;
myRequestState.conn = this;
myRequestState.request = myHttpWebGetRequest;
myRequestState.request.ContentType = "application/json;charset=UTF-8";
myRequestState.request.Method = "POST";
myRequestState.request.AllowReadStreamBuffering = false;
myRequestState.request.Headers["UserName"] = "rick.labarbera#gmail.com";
myRequestState.request.Headers["Password"] = "125124514";
// myRequestState.request.Headers["MemberId"] = "UdE8IwmTbxEjmzmMo2nBpg==";
IAsyncResult result = (IAsyncResult)myHttpWebGetRequest.BeginGetResponse(new AsyncCallback(RespCallback), myRequestState);
}
catch (Exception e)
{
close(myRequestState);
if (this.listener != null)
{
Logger.log(TAG, "post()", e.Message);
}
}
}
public void Get()
{
RequestState myRequestState = new RequestState();
try
{
System.Uri uri = new Uri(url);
HttpWebRequest myHttpWebPostRequest;
Logger.log(TAG, "Create a HttpWebrequest object to the URL", url);
myHttpWebPostRequest = (HttpWebRequest)WebRequest.Create(uri);
_mHttpWebRequest = myHttpWebPostRequest;
myRequestState.conn = this;
myRequestState.request = myHttpWebPostRequest;
myRequestState.request.Method = "GET";
myRequestState.request.AllowReadStreamBuffering = false;
IAsyncResult result = (IAsyncResult)myHttpWebPostRequest.BeginGetResponse(new AsyncCallback(RespCallback), myRequestState);
}
catch (Exception e)
{
close(myRequestState);
if (this.listener != null)
{
Logger.log(TAG, "get()", e.Message);
}
}
}
Am i doing something wrong???All these things are very very new to me.. I need help badly..
Thanks :)
I have played a bit with your code, but couldn't make it :(
What are the URL's you are using for the POST() method and for GET() methods.?
By the way, There is another way around to invoke your service. Follow these steps:
-- Create a new project.
-- Right-click on the Project name and click on "Add Service Reference"... Then provide address as "http://geniewebsvc.cloudapp.net/Member.svc" and click Go.
-- Once service information is downloaded, provide Namespace something like "MyMemberService" and click Ok.
Then Goto your MainPage.xaml.cs and write the following code.
MemberServiceClient client = new MemberServiceClient();
client.AuthenticateCompleted += new EventHandler<AuthenticateCompletedEventArgs>(client_AuthenticateCompleted);
client.AuthenticateAsync(new AuthCredentials() { UserName = "rick.labarbera#gmail.com", Password = "125124514" });
And the AuthenticateCompleted handler is
void client_AuthenticateCompleted(object sender, AuthenticateCompletedEventArgs e)
{
MessageBox.Show(e.Result.Successful.ToString());
}
This way you can simply call any service in the MemberService with just 2 or 3 lines of code. This is how a soap client is invoked in a Visual Studio project.
But again, there are some "Endpoint configuration" issues in this which you need to solve. And if you can do that you can save atleast 30 to 40 % of your development time.
Good luck.