VBA and json objects - json

I want to create meail merge letters to members of a community, reminding them of the new year and to pay their fees. There is a nice API on a server that can create QR-codes that the a phone pay app can read. The server responds to HTTP POST rewuests. This API provides the possibility to pre-fill in the payment parameters like the payment receiver, the amount an a message to the payment receiver.
I am starting from an Excel-sheet that keeps the list of members amongst other items. I can create mail merge letters from this member list, but fails to get the creation of the prefilled QR-codes work correctly.
Since the base is Excel (and Word for the mail merge) the language used is VBA. With the code below i can get response from the server with a prefilled QR-code that contains the recieiver number of the payment:
Sub DownloadQRCode(ByVal myURL As String, ByVal LocalFileName As String)
Dim msXML As New MSXML2.ServerXMLHTTP60
Dim myStream As New ADODB.Stream
msXML.Open "POST", myURL, False
msXML.setRequestHeader "Content-Type", "application/json"
msXML.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"
msXML.send "{""payee"":""987654321""}"
If msXML.Status = 200 Then
myStream.Open
myStream.Type = adTypeBinary
myStream.Write (msXML.responseBody)
myStream.SaveToFile LocalFileName, adSaveCreateOverWrite
myStream.Close
End If
Set msXML = Nothing
Set myStream = Nothing
End Sub
If I change the msXML.send statement to
msXML.send "{""message"":{""value"":""Sven_Svensson"",""editable"":""false""}}"
I get the response code 400 (Bad request)
The speicification for the message object looks like this
My questions are: Do I use the wrong value in "Content-Type" header or is the json representation of the message object faulty?
SOLVED!
A couple of mistakes from my side:
Phone/account numbers must be valid in this application
Booleans must not be quoted
Numbers must not be quoted
json uses brackets for arrays, not as outer paranthesis
I have this working fine now

This works for me:
msXML.send "{""payee"":""987654321"",""message"":{""value"":""Sven_Svensson"",""editable"":true}}"
Note booleans in JSON are not quoted.

You used an array ([]) in your message 'msXML.send "{""message"":[{""value"":""Sven_Svensson"",""editable"":""false""}]}"' while message's specification doesn't state it. I guess it should look like 'msXML.send "{""message"":{""value"":""Sven_Svensson"",""editable"":""false""}}"'

Related

Unable to GET a value associated to div Tag <div class="name"> Value </div>

I'm using VBA to pull a value from a nested Div Tags, as shown on below image:
HTML code:
<div class="styles__SCFileModuleFooter-e6rbca-1 kUUNkj">
Constituída em 26-02-1962
</div>
I'm using the code Below to access class="styles__SCFileModuleFooter-e6rbca-1 kUUNkj", and pull the value "Constituída em 26-02-1962", but seems it is not capable to find that class in the loop:
Dim XMLPage As New MSXML2.XMLHTTP60
Dim HTMLDoc As New MSHTML.HTMLDocument
Dim htmlEle1 As MSHTML.IHTMLElement
Dim htmlEle2 As MSHTML.IHTMLElement
Dim URL As String
Dim elemValue As String
URL = "https://www.informadb.pt/pt/pesquisa/?search=500004064"
XMLPage.Open "GET", URL, False
XMLPage.send
HTMLDoc.body.innerHTML = XMLPage.responseText
For Each htmlEle1 In HTMLDoc.getElementsByTagName("div")
Debug.Print htmlEle1.className
If htmlEle1.className = "styles__SCFileModuleFooter-e6rbca-1 kUUNkj" Then
elemValue = Trim(htmlEle1.innerText)
If InStr(UCase$(elemValue), "CONSTITU") > 0 Then
'Found Value
Exit For
End If
End If
Next htmlEle1
There are some cases where the class="styles__SCFileModuleFooter-e6rbca-1 kUUNk" appears more than once, that's the reason for the checking If InStr(UCase$(elemValue), "CONSTITU") > 0 Then.
Can anyone help please?
You are not capturing what actually happens when you use a browser to make that search.
Among the various steps are the following ones:
You navigate to https://www.informadb.pt/pt/pesquisa/?search=500004064
Webpage generates additional traffic
Amongst that additional traffic is an API POST XHR request which returns search results as JSON. That request goes to https://www.informadb.pt/Umbraco/Api/Search/Companies and includes the 500004064 identifier amongst the arguments within the post body
Based on the API results the browser ends up at the following URI https://www.informadb.pt/pt/pesquisa/empresa/?Duns=453060832
You are not ending up at this same endpoint, where you would find the target class, as JavaScript does not run with your XMLHTTP request, and thus the additional traffic is not generated.
You might try recreating the POST request which returns the necessary Duns value to take you to the same endpoint. The POST request response is as follows:
{'TotalResults': 1,
'NumberOfPages': 1,
'Results': [{'Duns': '453060832',
'Vat': '500004064',
'Name': 'A PANIFICADORA CENTRAL EBORENSE, S.A.',
'Address': 'BAIRRO DE NOSSA SENHORA DO CARMO,',
'Locality': 'ÉVORA',
'OfficeType': 'HeadOffice',
'FoundIn': None,
'Score': 231.72766,
'PageUrl': '/pt/pesquisa/empresa/?Duns=453060832'}]}
You can see the necessary Duns value in the response. You would need to parse out this value possibly with a JSON parser.
However, a quick look at how the server responds to requests seems to indicate it expects to receive JSON and not simply as a string. This rules out typical XHR/WinHttp request examples you will find on the web, though you might have some luck with the implementation via https://github.com/VBA-tools/VBA-Web. However, I have found that library to constantly throw errors when I have tried using it.
With Python, for example, you can easily just POST a JSON object specifying json = data within the requests.post() call if using requests library.

Visual Basic - HTTPClient and API Request Issue

Good morning,
As a returning developer (I've been in management for a long time!), I've been tasked with developing a module for our proprietary financial system (Visual Basic/ SQL Server) that will read large .CSV files containing customer information, convert them into JSON format, then submit them to an external party for processing via that company's API.
The conversion part was easy and I'm almost ready to go with it, but I can't establish connectivity to the external API.
There are two parts to the process: -
Submit login details (obfuscated here) {"username": "MadeUpUser","password": "Y66***uYj6%%YY"} and receive a Bearer Token from the API
Submit JSON format customer info to API endpoint, using Bearer Token, receive confirmation
I've submitted both the login creds and my JSON-format data to the API via Postman and it works perfectly, however, when I try to login via my VB app using the HTTPClient class, I'm getting the follow exception (copied from the Exception class properties) :
**HResult = -2146233088
StackTrace = " at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context)" & vbCrLf & " at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)"
InnerException = {"The request was aborted: Could not create SSL/TLS secure channel."}**
It's clearly something I'm doing wrong programmatically or something I'm not doing and any help or advice offered will be greatly appreciated.
The code: (AddLogEntry() is a Sub I created to log events to a text file)
Private Async Function PostAsync(ByVal JSONString As String, EndPoint As String) As Task
Dim APIuri As New Uri(EndPoint)
Try
With ZincClient
.BaseAddress = APIuri
.DefaultRequestHeaders.Accept.Clear()
.DefaultRequestHeaders.Accept.Add(New Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"))
End With
Dim content As New Net.Http.StringContent(JSONString, System.Text.Encoding.UTF8, "application/json")
Dim response As Net.Http.HttpResponseMessage = Await ZincClient.PostAsync(APIuri, content)
Dim result As String = Await response.Content.ReadAsStringAsync()
Catch ex As Exception
AddLogEntry("ERROR! - " & ex.Message)
End Try
Return
End Function

How do I get the body from an HttpWebResponse using Visual Basic (VS2017)

I am having a hell of a time with this project I am working on and could really use your help. I'll try to keep this as concise as possible.
Basically, I have a UI setup that collects user information and sends REST API calls to my call control system. Currently it is working in regards to adding lines, trunks etc.
The problem is I want to see the body of the response coming back from the server. If I have the exception handling look for an OK status code and turn an indicator green, it performs as I have asked.
But I cannot get it to give me the full body of the response like it does when I send the call in POSTMAN.
Here is the response I get back in POSTMAN and would like to get back when I make the API call in VB. I get a Status Code 400 with this in the body.
{
"hint": "The specified Pattern (2952) is already in use by the line owner.",
"details": "Conflicting Pattern",
"code": "P0001",
"message": "invalid_parameter" }
I have tried every combination of search criteria I could find on this site and nothing seems to be working for me. I just don't know what I am doing wrong. Here is a sample of how the request/response is set up now.
Try
'//Setup HTTP connection and modify headers
Dim APIRequest As HttpWebRequest = HttpWebRequest.Create("https://" & CMIP.Text & "/api/v1/" & Custom_Endpoint_URL.Text)
APIRequest.Method = "POST"
APIRequest.Headers.Add("Authorization", "Bearer " + LatestToken.Text)
APIRequest.ContentType = "application/json"
'WebCall.Headers.Add("Prefer", "return=representation") 'Only used for testing purposes
'//Prepare JSON request
Dim bytearray As Byte() = System.Text.Encoding.UTF8.GetBytes(Custom_Body.Text)
APIRequest.ContentLength = bytearray.Length
'//Bypass self-signed cert issue
ServicePointManager.ServerCertificateValidationCallback = AddressOf AcceptAllCertifications
'//Load JSON payload into datastream
Dim datastream As Stream = APIRequest.GetRequestStream()
datastream.Write(bytearray, 0, bytearray.Length)
''//Response
Dim response As WebResponse = APIRequest.GetResponse
Dim responsestream As Stream = response.GetResponseStream
Dim responsereader As New StreamReader(responsestream)
Dim responsereadstring As String = responsereader.ReadToEnd
'//Send response to results window
Response_Box.Text = responsereadstring
Catch ex As WebException
End Try
I wanted to figure this out on my own but I have been at this for a few days now and I'm at the point I am banging my head against the wall.
The HttpWebRequest.GetResponse method throws a WebException when the response has a status code in the 4xx or 5xx range. Based on the information in your question, your API call is returning a status code of 400, so that would trigger the exception. Your Catch block is completely empty, so you are silently ignoring this case. (That is why you should never leave your Catch blocks empty-- you'll never know that something is failing.)
It turns out that the WebException class has a Response property on it (as well as Status), so you can get the data from there. You just need to fill in the Catch block:
Catch ex As WebException
Dim responsestream As Stream = ex.Response.GetResponseStream
Dim responsereader As New StreamReader(responsestream)
Dim responsereadstring As String = responsereader.ReadToEnd
Response_Box.Text = responsereadstring
End Try

Logging into websites using JSoup, is it possible?

Is it possible to log into websites that use csrf tokens and such using JSOUP? The website I am trying to log into is aliexpress.com. Which seems to have a lot of input values, and I noticed that the csrf token changes on every attempt. I am guessing that these are protective measures to block out spam. I am quite new to html, and was wondering, is it even possible to login to sites like these? Thanks
Here is my code in case I did something wrong:
public static void main(String[] args) throws IOException {
Connection.Response loginForm = Jsoup.connect("http://login.aliexpress.com")
.method(Method.POST)
.data("cookieexists", "false")
.data("loginId", "xxxxxx#gmail.com")
.data("password", "xxxxxx")
.data("event_submit_do_login", "submit")
.data("submit-btn", "Sign In")
.data("appName", "aebuyer")
.data("appEntrance", "default")
.data("_csrf_token", "vdspQZH4cMoQT6GxLyU0a7")
.data("rdsToken", "")
.data("cid", "a832dd6d-990f-44eb-8bdb-9ec49d1c0a99")
.data("umidToken", "4e6219e38c34346dc2bb7914a54794aac7645e7b")
.data("lang", "en_us")
.data("hsid", "VP4zHOZfVs1Ec4qqEsI1mA")
.data("isRDSReady", "false")
.data("isUMIDReady", "false")
.data("umidGetStatusVal", "")
.data("bizParams", "")
.data("isRequiresHasTimeout", "false")
.data("loginHost", "https://passport.aliexpress.com/")
.data("scene", "")
.data("isMobile", "false")
.data("modulus", "d3bcef1f00424f3261c89323fa8cdfa12bbac400d9fe8bb627e8d27a44bd5d59dce559135d678a8143beb5b8d7056c4e1f89c4e1f152470625b7b41944a97f02da6f605a49a93ec6eb9cbaf2e7ac2b26a354ce69eb265953d2c29e395d6d8c1cdb688978551aa0f7521f290035fad381178da0bea8f9e6adce39020f513133fb")
.data("exponent", "10001")
.data("ua","099#KAFEx7E+TEGE6YTLEEEEE6twSXLoZ6NHDu3YS6tqDywMC6t3gR9YLfD1SryM4MDFYuJoV6t1YRB7n6f1Ych6n6g1DRJo+xqTETFUEcZt288mby1PYPrfds1gLjdTEEi5DEEErGFEHCLlhRaTG9llEJIm/KRlbpZV8MixZy164GFEJFwlsyaDMOcmby1TSV3WPy1tY0s3YuaHLyZTvbd9E7EFD67EEKqTETAEluZdtkH7NyhGYfciuOkG/MhXaQ3bAMZtY0s3JGFET6i5EE1iE7Eo73lP/3m4NVkAbIJi4leDbRZTvbuwsqk6/MhXaCqTETfEEcZt9CXmby1PYPrfds1tYo8XCV/zLybTETDElapdLIallGEeOOCxls1Pr08FSP8zYovKZMRc41qTETFUEcZt9Scmby1PYPrfds1gLioTETzElsXdLUQEWECmYKhMtV8gPy1fnM9SZyvHLioTETYQldXd7FEEVcCmYKhMtV8gPy1XC0ysUoSnE7EiK3l6/wqaEcExOMRyv9cGNyvHYp5TEEi5D7EE/GFETJDovl8JE7hMDwGEmba3U61vv0XBaYevDpiRciUt1OLjbo4Qc7rczQ8nZR2Bw7CWSQ/vPsqqi9exUqZ96Yz6bQWAnWuWLLrDA2iebRMREyB7Pbcdb4QsPok4QioWiYeWZyhqqOKR1eJ6UI2Vcgw6bXkPnypX1YuYalp7Pwnh1eyJroeXFSZ6UcCsZLpzLUZ1WsJhvLqpa291dzCncU7RbohMtscSi7hYSeJFFsxcTy/0/fhCv4bprqrANa2n+GCWds166OIp+WJFrtSSPFnRrVs8SAUM+GZDnR+kbyo6cWyIbISMHG1uVPybZz4M/vl4DXym8jvqaWw7zVZBNdZ6bteWNREZwErbnpywry4t+O/kuV8W/EPcuzpYa0uVaYnzZuB5hg9ulrimV65MJwnuV6i4DTY03BlWaccJ8ypu9u2A8WWzPMIszbcYDi4nEoCYDpiB4gvl9zkGVPq0IlbuQfcxVyrs37eQSyJKcw4RLeXKPt+Z4LslVewCYREnCQZVgcx+hz+l/ri5Vfx+8IRuNP94S0Yo3/sLDVwjvK4l7uyxOGFETYilssn1dASTEELlluaLom7wxeSTE1LlluZdt3illllls3aSt3t3llle33iSw6alluUdt37q33llWLaStEGde7FE5YHKJ7ThVa6o3kskwmyV0HUC6Df39yzU0xEG1DN7Vafo3kikLw+IRhj2qbAB6OT2qago1DN7VaTo3kikawWIRhj2qoAB6OT2qafo1DN7VmGtI8EJE7EhlAaP/3iSJGFET6i5EE1mE7EFlllbrxqTETFUluZdDWkmby1PYPrfds1gLioTETYQltCdBHGEV38mYKhMtV8gPy1XC0ysUoSnE7EB63l7/h6LEc7JNM1j83X0UykG+qxEdUF9E7EFD67EE1qTETFUluZt9CZmby1PYPrfds1gLjdTEEi5DEEE4GFEJFwlsyaD0Lxmby1TSV3WPy1tY0s3YuaHLyZTvbuiE7Eo73lP/3m+q0kAbIJi4leDbRZTvbuwsqk6/MhXaVqTETAEluZdtkpCNyhGYfciuOkG/MhXaQ3bAMZtY0s3IGFEHSwlsyDe2VkAbpZ1ZVnabRZTC0sR8u86")
.followRedirects(true)
.execute();
// TODO code application logic here
Document doc = loginForm.parse();
System.out.print("website title:" + doc);
}
}
Since you know which parametrers to send, I assume that you know how to use your browser's developer's tools, so it will be easy for you:
In order to login to the site, you have to take two steps. The first step is sending a GET request and parse the result. The second step is to send a POST request, with the needed parameters, including the ones you've obtained from step 1.
I've seen that when sending the first get request to https://login.aliexpress.com/, the result does not contain the values of _csrf_token etc. The browser sends another request to https://passport.aliexpress.com/mini_login.htm?lang=en_us&appName=aebuyer&appEntrance=default&styleType=auto&bizParams=&notLoadSsoView=false&notKeepLogin=true&isMobile=false&rnd=0.9476178801629621 so you must do the same, and parse the result (notice that the last parameter of the get request is some random number. I think you should also randomize the string and not use the same number again and again, it might be some protection measure):
String firstURL = "https://passport.aliexpress.com/mini_login.htm?lang=en_us&appName=aebuyer&appEntrance=default&styleType=auto&bizParams=&notLoadSsoView=false&notKeepLogin=true&isMobile=false&rnd=0.9476178801629621";
Connection.Response loginForm = Jsoup.connect(firstURL)
.userAgent("Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0")
.method(Method.GET);
After that you'll have to parse the response and extract the parameters, something like this -
Element e = doc.select("input[id=fm-cid]").first();
String cid = e.attr("value");
After parsing all the needed values, you can send the POST request.

Requesting URL with UTF-8 characters from VBA code

I am requesting results from this API:
http://mlrs.research.um.edu.mt/resources/gabra-api/
Everything works fine except when I introduce Maltese characters (which are UTF-8).
If I manually request the data using the following URLs, the return is correct.
http://mlrs.research.um.edu.mt/resources/gabra-api/lexemes/search?s=għar
...search?s=ċar (can't post more than two links yet.)
When using the following code, the return is blank.
{"results":[],"query":{"page":1,"page_size":20,"result_count":0,"term":"g?ar","search_lemma":true,"search_wordforms":true,"search_gloss":true,"pending":false,"pos":null,"source":null}}
{"results":[],"query":{"page":1,"page_size":20,"result_count":0,"term":"?ar","search_lemma":true,"search_wordforms":true,"search_gloss":true,"pending":false,"pos":null,"source":null}}
Note ? replacing ħ and ċ characters - that's only because I copied these from the immediate window.
This is the code I am using to make the requests:
Public Function GetWebSource(ByRef Url As String) As String
Dim xml As IXMLHTTPRequest
On Error Resume Next
Set xml = CreateObject("Microsoft.XMLHTTP")
With xml
.Open "GET", Url, False
.send
GetWebSource = .responseText
End With
Set xml = Nothing
End Function
Because VBA IDE does not support these characters, tests will need to be done from a form field.
Any help, much appreciated.
Thanks in advance.
Stephen
The URL contains some non ASCII characters, so you'll have to encode them beforehand:
Set xhr = CreateObject("Microsoft.XMLHTTP")
xhr.Open "GET", "http://mlrs.research.um.edu.mt/resources/gabra-api/lexemes/search?s=g%C4%A7ar", False
xhr.Send