On link Google drive link
I have value of JSON like
{
"APP_Name":"ThunderStorm",
"Version":"2.0.0.0",
"Download_URL":"PathToExe",
"Release_Info":"Info"
}
In my vb.net code when i use link to download that string
Dim client As New Net.WebClient
Dim Root_JSON As String = client.DownloadString("https://drive.google.com/open?id=1grin5l7VUa_Yw48H6ScFCKbMWv-Y-QpF")
I get bunch of HTML data. I need to download only JSON value so i would be able to deserialize it
Dim o_Json As t_Update.JSON_Application = JsonConvert.DeserializeObject(Of JSON_Application)(Root_JSON)
Related
We are using Azure APIM service & linked App Insights to get the log of request for analysis purpose.
As we are using Imperva WAF, we are getting Imperva IPs and getting client's IPs in request header thru App Insights Logs(Analytics).
So getting data in csv format, something like this
"2019-10-06T17:21:20.2264252Z","|key",,"OPTIONS /Endpoint/","https://APIM/query",True,200,"0.3565","<250ms",request,"{""ApimanagementServiceName"":""APIM_name"",""ApimanagementRegion"":""Country"",""HTTP Method"":""OPTIONS"",""API Name"":""API"",""Cache"":""None"",""Request-Incap-Client-IP"":""2a:23c4:1c4c:6c00:a458_IP""}","{""Client Time (in ms)"":0,""Response Size"":334,""Request Size"":0}","OPTIONS /EndPoint/",Key,,,,,,,PC,,,"0.0.0.0",,,"Country",,"APIM Country","APIM Country","key","App Insights","key","apim:0.81.21.0","key",1
Now i want to extract IP from "Request-Incap-Client-IP" element which is stored in JSON format starting from "ApimanagementServiceName".
I looked at help on web and all are talking about macro, custom code.
In my opinion, excel should have function to parse json and get value from specific columns i mean solution should be straightforward and simple.
Use that custom library to parse/read/write JSON files and streams
https://github.com/VBA-tools/VBA-JSON
Dim fso As New FileSystemObject
Dim JsonTS As TextStream
Dim Json As Dictionary
Dim JsonText As String
Set JsonTS = fso.OpenTextFile("yourfile.csv", ForReading)
JsonText = JsonTS.ReadAll
JsonTS.Close
'process the string to isolate JSON part of your CSV using Split/regexp
Set Json = JsonConverter.ParseJson(JsonText)
Then you can just retrieve the value like that :
Dim ipValue as String
ipValue = Json("Request-Incap-Client-IP")
I have been trying to parse a bit of JSON data from an API as part of a larger project and due to my limited experience with VBA/Json I am having difficulties figuring out what the problem is with my VBA code.
I have used the following VBA Json Parsers and all basically stop working at the same point and give me an "incomplete parse".
vba-json https://code.google.com/archive/p/vba-json
vbjson www.ediy.co.nz/vbjson-json-parser-library-in-vb6-xidc55680
The Json I am trying to parse is the following:
{"links":[],"content":{"carrier":{"allowedToOperate":"Y","bipdInsuranceOnFile":"1000","bipdInsuranceRequired":"Y","bipdRequiredAmount":"750","bondInsuranceOnFile":"0","bondInsuranceRequired":"N","brokerAuthorityStatus":"N","cargoInsuranceOnFile":"0","cargoInsuranceRequired":"N","carrierOperation":{"carrierOperationCode":"A","carrierOperationDesc":"Interstate"},"commonAuthorityStatus":"A","contractAuthorityStatus":"A","crashTotal":19,"dbaName":null,"dotNumber":124871,"driverInsp":974,"driverOosInsp":33,"driverOosRate":3.38809034907597535934291581108829568789,"driverOosRateNationalAverage":"5.51","ein":460396676,"fatalCrash":0,"hazmatInsp":0,"hazmatOosInsp":0,"hazmatOosRate":0,"hazmatOosRateNationalAverage":"4.5","injCrash":6,"isPassengerCarrier":"N","issScore":null,"legalName":"A&A EXPRESS LLC","oosDate":null,"oosRateNationalAverageYear":"2009-2010","phone":"6055822402","phyCity":"BRANDON","phyCountry":"US","phyState":"SD","phyStreet":"1015 9TH AVENUE NORTH","phyZipcode":"57005","reviewDate":"2016-09-23","reviewType":"N","safetyRating":"S","safetyRatingDate":"2008-08-29","safetyReviewDate":"2016-09-23","safetyReviewType":"N","snapshotDate":"1485493200000","statusCode":"A","totalDrivers":237,"totalPowerUnits":253,"towawayCrash":13,"vehicleInsp":477,"vehicleOosInsp":58,"vehicleOosRate":12.15932914046121593291404612159329140461,"vehicleOosRateNationalAverage":"20.72"},"links"
my VBA code looks like the following:
Public Sub PARSEJSON()
Dim reader As New XMLHTTP60
Dim api As Object
Dim WS As Worksheet
Dim item As Variant
reader.Open "GET", "https://mobile.fmcsa.dot.gov/qc/services/carriers/124871?webKey=APIKEY", False
reader.send
Set WS = ActiveSheet
Set api = JSON.parse(reader.responseText)
For Each item In api
WS.Range("a3").Value = item("")
Next
End Sub
I cannot get into the JSON to reference the values that I need because the parser yields a result like:
photo of watcher in VBA
it seems like the Json parser is only breaking down part of the Json string from the API, it is from a government website so I am assuming it should be in a common Json format. I hope I included enough detail in this post to encompass the scope of my issue
I solved this issue, I looked further into how to reference a dictionary and realized I was using the wrong syntax to reference and adding in extra steps that were not needed.
new VBA code yields the results I want:
Public Sub PARSEJSON()
Dim reader As New XMLHTTP60
Dim dict As Object
Dim api As New Scripting.Dictionary
Dim json As New Class1
Dim WS As Worksheet
Dim key As Variant
Dim item As Variant
reader.Open "GET", "https://mobile.fmcsa.dot.gov/qc/services/carriers/124871?webKey=APIKEY", False
reader.send
Set WS = ActiveSheet
Set api = json.parse(reader.responseText)
MsgBox api("content")("carrier").Count
MsgBox api.item("content")("carrier")("driverOosInsp")
WS.Range("a1").value = api.item("content")("carrier")("driverOosInsp")
End Sub
Flow:
1. (OK) i download a json
2. (OK) i parse a value from the json object that contains html
3. (NOT OK) i display the values inside div.countries
my code:
Dim webClient As New System.Net.WebClient
Dim result As String = webClient.DownloadString("http://example.com/countries.json")
Dim values As JObject = JObject.Parse(result)
Dim finalHTML As String = values.GetValue("countries_html")
basically finalHTML variable looks like this:
<div class="country_name">USA</div>
<div class="country_name">Ireland</div>
<div class="country_name">Australia</div>
Im stuck and dont know how to move on.
I need to go over all div.country_name and get the inner_text of it. Hope that make sense.
Since the finalHTML string already contain only the target div elements, you can simply load the string to HtmlDocument object and use a bit of LINQ to project the divs into collection -IEnumerable, List<T>, or whatever most suitable to your need- of InnerText strings :
....
Dim finalHTML As String = values.GetValue("countries_html")
Dim doc = New HtmlDocument()
doc.LoadHtml(finalHTML)
Dim countries = doc.DocumentNode.Elements("div").Select(Function(o) o.InnerText.Trim())
'print the result as comma separated text to console:
Console.WriteLine(String.Join(",", countries))
Dotnetfiddle Demo
output :
USA,Ireland,Australia
here's a nice article on using the HAP: http://www.mikesdotnetting.com/article/273/using-the-htmlagilitypack-to-parse-html-in-asp-net.
I am using lotus notes form as .html files and I am sending values to server as json using angular js. But I want to upload files also now. How can I send files to server and extract using lotus script?
Can you please help me someone?
Like the below post. But it is done in ASP.NET . I want to do the same using lotus notes.
File uploading angular js ASP .NET
index.html
<span ng-if="quests.type == '17'">
<input type="file" file-upload multiple id='{{quests.id}}'/>
</span>
<button type="button" ng-click="submitForm();">Submit</button>
The above button will trigger the below code to executed.
Angular Code to post to server
var email=document.getElementById("email").value;
var message={"requesttype": "saveForm","email": emailid,"username": username};
$http.post("http://test.com/ajaxprocess?openagent", message).success(success).error(failure);
The above mentioned agent(lotusscript) will parse the above json and save the document as shown below.
ajaxprocess Agent code
'getting document context
Set docContext = sess.DocumentContext
If docContext.hasItem("REQUEST_CONTENT") Or docContext.hasItem("REQUEST_CONTENT_000") Then
'using openNTF lotus script classes to parse document to json object
Set userDataInfo=getJSONObjectFromDocument(docContext, "")
Dim fieldsobj As New JSONArray
'getting the fields array sent as json array
Set fieldsobj=userDataInfo.GetItemValue("fields")
fieldtype=Field.mGetItemValue("type")(0)
Dim doc As NotesDocument
Dim fieldname As String
ForAll Field In fieldsobj.Items
fieldname=Field.mGetItemValue("Fieldname")(0)
Call doc.Replaceitemvalue(fieldname,Field.mGetItemValue("value")(0))
End ForAll
call doc.save(true,false)
End If
Everything works fine expect file attachments. How can I send files to server with json and save using lotus script or is there any other workaround is there?
I finally found tip and made the solution as follows to get the base64 String and convert to attachment in lotusscript.
http://www-10.lotus.com/ldd/bpmpblog.nsf/dx/creating-a-mime-email-with-attachment?opendocument&comments
Dim s As New NotesSession
Dim stream As NotesStream
Dim body As NotesMIMEEntity
Dim header As NotesMIMEHeader
Dim StringInBase64 As String
StringInBase64=getbase64() 'your base64 string
Dim db As NotesDatabase
Set db=s.Currentdatabase
Dim tempdoc As NotesDocument
Set tempdoc=db.Createdocument()
Set stream = s.CreateStream
Call stream.WriteText(StringInBase64)
Set body = tempdoc.CreateMIMEEntity
Set header = body.createHeader("content-disposition")
Call header.setHeaderVal({attachment;filename="Onchange.xlsx"}) ' file name and type should be configurable
Call body.SetContentFromText(stream, "", ENC_BASE64)
Call stream.Close
tempdoc.form="Attachment"
Call tempdoc.save(True,False)
This works as expected. Thanks all for time you spent.
Here is the code for Multiple attachments, enhancement from Vijayakumar.
Dim session As New NotesSession
Dim db As NotesDatabase
Dim doc As NotesDocument
Set db = session.CurrentDatabase
Set doc = db.CreateDocument
Dim s As New NotesSession
Dim stream As NotesStream
Dim body As NotesMIMEEntity
Dim child As NotesMimeEntity
Dim header As NotesMIMEHeader
Set body = doc.CreateMIMEEntity
topString = Split(BASE64, ",")
Dim tmp_array() As String
i = 0
For i = 0 To Ubound(topString)
Redim Preserve tmp_array(i)
tmp_array(i) = topString(i)
Set child = body.CreateChildEntity()
Set header = child.CreateHeader("Content-Type")
Call header.SetHeaderVal("multipart/mixed")
Set header =child.createHeader("Content-Disposition")
Call header.setHeaderVal({attachment; filename=test} &Cstr(i)& {.jpg}) 'file name and type should be configure
Set header =child.CreateHeader("Content-ID")
Call header.SetHeaderVal("test" &Cstr(i)& ".jpg")
Set stream = s.CreateStream()
Call stream.WriteText(topString(i))
Call child.SetContentFromText(stream, "", ENC_BASE64)
Next
doc.form="Attachment"
'doc.Attachment = tmp_array
Call doc.save(True,False)
Call stream.Close()
s.ConvertMIME = True ' Restore conversion
I must write a program that periodically reads a web page and copies certain data from a table on that page to an Excel spreadsheet. I don't know where to start or what programming language is suitable for this project. I know a little C++ and Matlab programming. Can anyone offer advice to point me in the right direction or suggest open source projects which do something similar?
I can use wget(linux) or fget1(matlab) to download the webpages, but I don't know how can I save certain data from source of this webpages into Excel.
I will assume you have room for learning C#. Since you have to extract the table from a web page, you need a special library/framework to deal with web browsing such as Watin. After getting the table, it's matter of saving into Excel spreadsheet. For convenience, you can write a CSV format (comma separated text) and excel can open the file. Hope it helps
I used the following code vb.net to parse multiple html table from a saved web page to a datatable (the table must have the same structure) (using Html-Agility-Pack) and save it to Xml file:
Imports System.Net
Public Sub ParseHtmlTable(byval HtmlFilePath as String)
Dim webStream As Stream
Dim webResponse = ""
Dim req As FileWebRequest
Dim res As FileWebResponse
req = WebRequest.Create("file:///" & HtmlFilePath)
req.Method = "GET" ' Method of sending HTTP Request(GET/POST)
res = req.GetResponse ' Send Request
webStream = res.GetResponseStream() ' Get Response
Dim webStreamReader As New StreamReader(webStream)
Dim htmldoc As New HtmlAgilityPack.HtmlDocument
htmldoc.LoadHtml(webStreamReader.ReadToEnd())
Dim nodes As HtmlAgilityPack.HtmlNodeCollection = htmldoc.DocumentNode.SelectNodes("//table/tr")
Dim dtTable As New DataTable("Table1")
Dim Headers As List(Of String) = nodes(0).Elements("th").Select(Function(x) x.InnerText.Trim).ToList
For Each Hr In Headers
dtTable.Columns.Add(Hr)
Next
For Each node As HtmlAgilityPack.HtmlNode In nodes
Dim Row = node.Elements("td").Select(Function(x) x.InnerText.Trim).ToArray
dtTable.Rows.Add(Row)
Next
dtTable.WriteXml("G:\1.xml", XmlWriteMode.WriteSchema)
End Sub
After that import the file to Excel
Read this Article to import XML into excel
Hope it helps