We are using Azure APIM service & linked App Insights to get the log of request for analysis purpose.
As we are using Imperva WAF, we are getting Imperva IPs and getting client's IPs in request header thru App Insights Logs(Analytics).
So getting data in csv format, something like this
"2019-10-06T17:21:20.2264252Z","|key",,"OPTIONS /Endpoint/","https://APIM/query",True,200,"0.3565","<250ms",request,"{""ApimanagementServiceName"":""APIM_name"",""ApimanagementRegion"":""Country"",""HTTP Method"":""OPTIONS"",""API Name"":""API"",""Cache"":""None"",""Request-Incap-Client-IP"":""2a:23c4:1c4c:6c00:a458_IP""}","{""Client Time (in ms)"":0,""Response Size"":334,""Request Size"":0}","OPTIONS /EndPoint/",Key,,,,,,,PC,,,"0.0.0.0",,,"Country",,"APIM Country","APIM Country","key","App Insights","key","apim:0.81.21.0","key",1
Now i want to extract IP from "Request-Incap-Client-IP" element which is stored in JSON format starting from "ApimanagementServiceName".
I looked at help on web and all are talking about macro, custom code.
In my opinion, excel should have function to parse json and get value from specific columns i mean solution should be straightforward and simple.
Use that custom library to parse/read/write JSON files and streams
https://github.com/VBA-tools/VBA-JSON
Dim fso As New FileSystemObject
Dim JsonTS As TextStream
Dim Json As Dictionary
Dim JsonText As String
Set JsonTS = fso.OpenTextFile("yourfile.csv", ForReading)
JsonText = JsonTS.ReadAll
JsonTS.Close
'process the string to isolate JSON part of your CSV using Split/regexp
Set Json = JsonConverter.ParseJson(JsonText)
Then you can just retrieve the value like that :
Dim ipValue as String
ipValue = Json("Request-Incap-Client-IP")
Related
I am having trouble sending JSON data to a firebase database using the rest API, the data is sent, but it does not parse. For instance if I use this curl command in command prompt in windows:
curl -X PUT -d "{\"lastName\":\"Jones\",\"firstName\":\"Bubba\"}" https://<database-name>.firebaseio.com/rest/test/.json
That results in the correct parsing of the data:
Yet, when using the following VBA code:
Sub PUSHhttpRequestTest() 'Doesn't Work!!
Dim sc As Object
Set sc = CreateObject("ScriptControl")
sc.Language = "JScript"
Dim strURL As String: strURL = "https://<database-name>.firebaseio.com/rest/.json"
Dim strRequest
strRequest = """{\""lastName\"":\""Jones\"",\""firstName\"":\""Bubba\""}"""
Dim XMLhttp: Set XMLhttp = CreateObject("msxml2.xmlhttp")
Dim response As String
Debug.Print strRequest
XMLhttp.Open "PUT", strURL, False
XMLhttp.setrequestheader "Content-Type", "application/json;charset=UTF-8"
XMLhttp.sEnd strRequest
response = XMLhttp.responseText
Debug.Print response
End Sub
This sends exactly the same stringified JSON, and it gets added to the Firebase database, however, the JSON string doesn't get parsed:
I have tried different Content Types, and variations on the JSON string, but nothing seems to work. Can anyone explain how I can get the VBA script to send data that Firebase will parse?
Thanks
I found a possible solution to sending JSON data from excel to firebase, but it doesn't answer my question about why the above VBA code sending a Stringified JSON doesn't get parsed in Firebase. I would still like a solution to that, because I already have a function the creates the stringified JSON from my data.
Using the VBA-web Library from this Stack Overflow post seems to do the trick. The example uses dictionaries for your data, however please my comment and the subsequent reply regarding the format of the JSON string to send. No escape code is required!
There is no PUT, and Other request types for json, but you can easily add these in yourself.
The equivalent code to the above, but using VBA-web library (with custom PutJson function) is:
Sub test()
Dim strURL As String: strURL = "https://<database-name>/rest/test/whatwhat/.json"
Dim strRequest As String: strRequest = "{""LastName"":""Jones"",""firstName"":""Bubba""}"
Dim Client As New WebClient
Dim Response As WebResponse
Set Response = Client.PutJson(strURL, strRequest)
ActiveSheet.Range("A1").Value = Response.Content
End Sub
And we end up with this....
Happy Days!
However, I'd still like to know why the seemingly identical curl and VBA HTTP requests result in different parsing of the data in FireBase?
On link Google drive link
I have value of JSON like
{
"APP_Name":"ThunderStorm",
"Version":"2.0.0.0",
"Download_URL":"PathToExe",
"Release_Info":"Info"
}
In my vb.net code when i use link to download that string
Dim client As New Net.WebClient
Dim Root_JSON As String = client.DownloadString("https://drive.google.com/open?id=1grin5l7VUa_Yw48H6ScFCKbMWv-Y-QpF")
I get bunch of HTML data. I need to download only JSON value so i would be able to deserialize it
Dim o_Json As t_Update.JSON_Application = JsonConvert.DeserializeObject(Of JSON_Application)(Root_JSON)
I have been trying to parse a bit of JSON data from an API as part of a larger project and due to my limited experience with VBA/Json I am having difficulties figuring out what the problem is with my VBA code.
I have used the following VBA Json Parsers and all basically stop working at the same point and give me an "incomplete parse".
vba-json https://code.google.com/archive/p/vba-json
vbjson www.ediy.co.nz/vbjson-json-parser-library-in-vb6-xidc55680
The Json I am trying to parse is the following:
{"links":[],"content":{"carrier":{"allowedToOperate":"Y","bipdInsuranceOnFile":"1000","bipdInsuranceRequired":"Y","bipdRequiredAmount":"750","bondInsuranceOnFile":"0","bondInsuranceRequired":"N","brokerAuthorityStatus":"N","cargoInsuranceOnFile":"0","cargoInsuranceRequired":"N","carrierOperation":{"carrierOperationCode":"A","carrierOperationDesc":"Interstate"},"commonAuthorityStatus":"A","contractAuthorityStatus":"A","crashTotal":19,"dbaName":null,"dotNumber":124871,"driverInsp":974,"driverOosInsp":33,"driverOosRate":3.38809034907597535934291581108829568789,"driverOosRateNationalAverage":"5.51","ein":460396676,"fatalCrash":0,"hazmatInsp":0,"hazmatOosInsp":0,"hazmatOosRate":0,"hazmatOosRateNationalAverage":"4.5","injCrash":6,"isPassengerCarrier":"N","issScore":null,"legalName":"A&A EXPRESS LLC","oosDate":null,"oosRateNationalAverageYear":"2009-2010","phone":"6055822402","phyCity":"BRANDON","phyCountry":"US","phyState":"SD","phyStreet":"1015 9TH AVENUE NORTH","phyZipcode":"57005","reviewDate":"2016-09-23","reviewType":"N","safetyRating":"S","safetyRatingDate":"2008-08-29","safetyReviewDate":"2016-09-23","safetyReviewType":"N","snapshotDate":"1485493200000","statusCode":"A","totalDrivers":237,"totalPowerUnits":253,"towawayCrash":13,"vehicleInsp":477,"vehicleOosInsp":58,"vehicleOosRate":12.15932914046121593291404612159329140461,"vehicleOosRateNationalAverage":"20.72"},"links"
my VBA code looks like the following:
Public Sub PARSEJSON()
Dim reader As New XMLHTTP60
Dim api As Object
Dim WS As Worksheet
Dim item As Variant
reader.Open "GET", "https://mobile.fmcsa.dot.gov/qc/services/carriers/124871?webKey=APIKEY", False
reader.send
Set WS = ActiveSheet
Set api = JSON.parse(reader.responseText)
For Each item In api
WS.Range("a3").Value = item("")
Next
End Sub
I cannot get into the JSON to reference the values that I need because the parser yields a result like:
photo of watcher in VBA
it seems like the Json parser is only breaking down part of the Json string from the API, it is from a government website so I am assuming it should be in a common Json format. I hope I included enough detail in this post to encompass the scope of my issue
I solved this issue, I looked further into how to reference a dictionary and realized I was using the wrong syntax to reference and adding in extra steps that were not needed.
new VBA code yields the results I want:
Public Sub PARSEJSON()
Dim reader As New XMLHTTP60
Dim dict As Object
Dim api As New Scripting.Dictionary
Dim json As New Class1
Dim WS As Worksheet
Dim key As Variant
Dim item As Variant
reader.Open "GET", "https://mobile.fmcsa.dot.gov/qc/services/carriers/124871?webKey=APIKEY", False
reader.send
Set WS = ActiveSheet
Set api = json.parse(reader.responseText)
MsgBox api("content")("carrier").Count
MsgBox api.item("content")("carrier")("driverOosInsp")
WS.Range("a1").value = api.item("content")("carrier")("driverOosInsp")
End Sub
I'm looking for a solution to automatically update a cell in my Excel Table via a JSON Webservice. I've found a way to do this via XML, but unfortunately no way to do this via JSON.
The Webservice I want to use is: https://mtgox.com/api/1/BTCUSD/ticker
And I would like to use the value of return.avg.value in my table.
I even thought of using RegEx to get the value but I could't find a way to do so, also.
I really hope there is a solution to my problem without using a lot of different Addons and Plugins for Excel, so my table remains portable.
Thank you for your help,
Frederick
Sub TestJson()
Dim json As String
Dim sc As Object, o
json = WebResponse("https://mtgox.com/api/1/BTCUSD/ticker")
Set sc = CreateObject("ScriptControl")
sc.Language = "JScript"
o = sc.Eval("eval(" & json & ")['return'].avg.value")
Debug.Print o
End Sub
Public Function WebResponse(sURL As String) As String
Dim XmlHttpRequest As Object
Set XmlHttpRequest = CreateObject("Msxml2.ServerXMLHTTP.4.0")
XmlHttpRequest.Open "GET", sURL, False
XmlHttpRequest.send
WebResponse = XmlHttpRequest.responseText
End Function
I must write a program that periodically reads a web page and copies certain data from a table on that page to an Excel spreadsheet. I don't know where to start or what programming language is suitable for this project. I know a little C++ and Matlab programming. Can anyone offer advice to point me in the right direction or suggest open source projects which do something similar?
I can use wget(linux) or fget1(matlab) to download the webpages, but I don't know how can I save certain data from source of this webpages into Excel.
I will assume you have room for learning C#. Since you have to extract the table from a web page, you need a special library/framework to deal with web browsing such as Watin. After getting the table, it's matter of saving into Excel spreadsheet. For convenience, you can write a CSV format (comma separated text) and excel can open the file. Hope it helps
I used the following code vb.net to parse multiple html table from a saved web page to a datatable (the table must have the same structure) (using Html-Agility-Pack) and save it to Xml file:
Imports System.Net
Public Sub ParseHtmlTable(byval HtmlFilePath as String)
Dim webStream As Stream
Dim webResponse = ""
Dim req As FileWebRequest
Dim res As FileWebResponse
req = WebRequest.Create("file:///" & HtmlFilePath)
req.Method = "GET" ' Method of sending HTTP Request(GET/POST)
res = req.GetResponse ' Send Request
webStream = res.GetResponseStream() ' Get Response
Dim webStreamReader As New StreamReader(webStream)
Dim htmldoc As New HtmlAgilityPack.HtmlDocument
htmldoc.LoadHtml(webStreamReader.ReadToEnd())
Dim nodes As HtmlAgilityPack.HtmlNodeCollection = htmldoc.DocumentNode.SelectNodes("//table/tr")
Dim dtTable As New DataTable("Table1")
Dim Headers As List(Of String) = nodes(0).Elements("th").Select(Function(x) x.InnerText.Trim).ToList
For Each Hr In Headers
dtTable.Columns.Add(Hr)
Next
For Each node As HtmlAgilityPack.HtmlNode In nodes
Dim Row = node.Elements("td").Select(Function(x) x.InnerText.Trim).ToArray
dtTable.Rows.Add(Row)
Next
dtTable.WriteXml("G:\1.xml", XmlWriteMode.WriteSchema)
End Sub
After that import the file to Excel
Read this Article to import XML into excel
Hope it helps