Streamreader to String not working properly - json

I am getting HttpWebResponse encoded in Base64
following lines get the webresponse from API.
Dim myResp As HttpWebResponse = myReq.GetResponse()
Dim myreader As New System.IO.StreamReader(myResp.GetResponseStream)
the response which i get is something like following, however actual response is too long and i cannot paste here so i have manually stripped the actual response.
{"status":"1","data":"eyJiMmIiOlt7ImludiI6W3siaXRtcyI6W3sibnVtIjoxODAxLCJpdG1fZGV0Ijp7ImNzYW10IjowLCJzYW10Ijo4MDkuOTEsInJ0IjoxOCwidHh2YWwiOjg5OTksImNhbXQiOjgwOS45MX19XSwidmFsIjoxMDYxOC44MiwiaW52X3R5cCI6IlIiLCJwb3MiOiIyNCIsImlkdCI6IjExLTA3LTIwMTgiLCJyY2hyZyI6Ik4iLCJpbnVtIjoiUldHSjA3LzE4LzAwMDU4NCIsImNoa3N1bSI6IjVjMjNiY2M1ZTQ3ZDI0NjU5YWQzNTEzNTM1YjhiNTAzNmM4NGU0MzU5NWJiMTVjYzA4M2VkYzBiNTQzZTQ1MzcifSx7Iml0bXMiOlt7Im51bSI6MTgwMSwiaXRtX2RldCI6eyJjc2FtdCI6MCwic2FtdCI6NDE4LjUsInJ0IjoxOCwidHh2YWwiOjQ2NTAsImNhbXQiOjQxOC41fX1dLCJ2YWwiOjU0ODcsImludl90eXAiOiJSIiwicG9zIjoiMjQiLCJpZHQiOiIyNS0wNy0yMDE4IiwicmNocmciOiJOIiwiaW51bSI6IlJXR0owNy8xOC8wMDEyNjEiLCJjaGtzdW0iOiJjOGEyMjNmNmMzYjY5ODZiYzE2MmNjYjdmMDhlZTYxMTdjYTdkOWZhNmEzYTExMWY1MmVjNzllYmExMGM5MWQ3In1dLCJjZnMiOiJZIiwiY3RpbiI6IjI0QUFCQ1I3MTc2QzFaSiJ9LHsiaW52IjpbeyJpdG1zIjpbeyJudW0iOjEsIml0bV9kZXQiOnsiY3NhbXQiOjAsInNhbXQiOjMzNzUsInJ0IjoxOCwidHh2YWwiOjM3NTAwLCJjYW10IjozMzc1fX1dLCJ2YWwiOjQ0MjUwLCJpbnZfdHlwIjoiUiIsInBvcyI6IjI0IiwiaWR0IjoiMzEtMDctMjAxOCIsInJjaHJnIjoiTiIsImludW0iOiJULTAxNzcvMjAxOC0xOSIsImNoa3N1bSI6ImYzNzFmYjA0N2FjNTRlOTkwYzZjNzM5Zjk0NTgwMzZlMWQxNjE0N2IxYmQ0ZTkxY2FlNmEwN2IyOGVlYzE0YWUifV0sImNmcyI6IlkiLCJjdGluIjoiMjRBQURDSTIwMzJFMVo5In1dfQ=="}
I am not sure why above Base64 Encoded message starts with {"status":"1","data":" and then ends with "}.
Actual Base64 data starts after {"status":"1","data":"
Due to those unsupported characters at starting and ending of the stream , i first try to convert actual response to string as shown below.
Dim myResp As HttpWebResponse = myReq.GetResponse()
Dim myreader As New System.IO.StreamReader(myResp.GetResponseStream)
Actual stream response returns around 248000 characters (as per response received in POSTMAN with same API). Streamreader information in Debug mode also shows same 248000 number. But when i convert them into string with following code line, string gets slimmed to around only 32000 characters. I don't know why this is happening?
Dim myText As String = myreader.ReadToEnd
'''Then following code will remove all those unwanted characters from starting string, which are {"status":"1","data":"
Dim Final_text As String = myText.Substring(myText.Substring(0, myText.LastIndexOf("""")).LastIndexOf("""") + 1)
'''Following code will remove two characters "} from end of the string.
Final_text = Final_text.Trim().Remove(Final_text.Length - 2)
''' Now Decode this proper Base64 String to JSON format
Dim data As Byte() = Convert.FromBase64String(Final_text)
Dim decodedString As String = Encoding.UTF8.GetString(data)
Dim JsonP As JObject = JObject.Parse(decodedString)
Dim SetPointerOut As JToken = JsonP("b2b")
Two things: why converting from Stream to String cut down actual response? 248000 charters to just apprx. 32000 characters. In debug mode if i type in ?mytext.length it returns 248000 as value. But When i hover mouse and brows what is in mytext variable, it shows me around 32000 charters only.
Service provider says Response which i get from API is Base64 encoded and i have to decode it before using it as JSON. Then why do i get unsupported characters at starting of the stream (even in Postman), is it Base64 Encoded message in serialized manner?
Am I doing right process to first convert the stream to string, remove unwanted characters and then Decode it? or there is some other way around.

Ok, issue of 32768 character in debug mode of Visual Studio is it self.
VS2015 had bug in which it does not support more than 32768 characters. Read
Why strings are shown partially in the Visual Studio 2008 debugger?
and
Visual Studio Text Visualizer missing text
The method which i was using to remove extra unwanted characters from "mytext" string, still works and give result. But as #Steve suggested in comment to the question, I should parse the JSON string. I find that idea much better and correct method.
so final code is like below:
Dim myResp As HttpWebResponse = myReq.GetResponse()
Dim myreader As New System.IO.StreamReader(myResp.GetResponseStream)
Dim myText As String = myreader.ReadToEnd
Dim json As String = myText
Dim jsonResult = JsonConvert.DeserializeObject(Of Dictionary(Of String, Object))(json)
Dim jsonObject As Newtonsoft.Json.Linq.JObject = Newtonsoft.Json.Linq.JObject.Parse(json)
Dim jsonValue As JValue = jsonObject("data")
Dim Final_text As String = jsonValue.ToString
''' No need of following code as doing JSON parse above
''' Dim Final_text As String = myText.Substring(myText.Substring(0, myText.LastIndexOf("""")).LastIndexOf("""") + 1)
'''Final_text = Final_text.Trim().Remove(Final_text.Length - 2)
Dim data As Byte() = Convert.FromBase64String(Final_text)
Dim decodedString As String = Encoding.UTF8.GetString(data)
Dim JsonP As JObject = JObject.Parse(decodedString)
Dim SetPointerOut As JToken = JsonP("b2b")

Related

Microsoft Cognitive API document size limit of 10240 bytes

When submitting a document to the API for key phrases, the returned JSON response has the error "A document within the request was too large to be processed. Limit document size to: 10240 bytes."
According to https://learn.microsoft.com/en-us/azure/cognitive-services/cognitive-services-text-analytics-quick-start, "The maximum size of a single document that can be submitted is 10KB, and the total maximum size of submitted input is 1MB. No more than 1,000 documents may be submitted in one call."
The document in question is a string of length 7713. The byte length using Encoding.UTF8.GetBytes() is 7763.
The entire byteArray that is submitted is of length 7965.
Smaller strings work fine, but any strings greater than length 3000 seem to have this problem. Below is the code, written in VB.NET:
' Create a JSONInput object containing the data to submit
Dim myJsonObject As New JSONInput
Dim input1 As New JSONText
input1.id = "1"
input1.text = text
myJsonObject.documents.Add(input1)
' Translate the JSONInput object to a serialized JSON string
Dim jss As New JavaScriptSerializer()
Dim jsonString As String = jss.Serialize(myJsonObject)
' Windows Cognitive Services URL
Dim request As System.Net.WebRequest = System.Net.WebRequest.Create("https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/keyPhrases")
' Set the Method property of the request to POST.
request.Method = "POST"
' Add a header with the account key.
request.Headers.Add("Ocp-Apim-Subscription-Key", accountKey_TextAnalytics)
' Create POST data and convert it to a byte array.
Dim postData As String = jsonString
Dim byteArray As Byte() = Encoding.UTF8.GetBytes(postData)
' Set the ContentType property of the WebRequest.
request.ContentType = "application/json"
' Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length
' Get the request stream.
Dim dataStream As System.IO.Stream = request.GetRequestStream()
' Write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length)
' Close the Stream object.
dataStream.Close()
' Get the response.
Dim response As System.Net.WebResponse = request.GetResponse()
' Get the stream containing content returned by the server.
dataStream = response.GetResponseStream()
' Open the stream using a StreamReader for easy access.
Dim reader As New System.IO.StreamReader(dataStream)
' Read the content.
Dim responseFromServer As String = reader.ReadToEnd()
' Display the content.
Console.WriteLine(responseFromServer)
' Clean up the streams.
reader.Close()
dataStream.Close()
response.Close()
' Deserialize the json data
jss = New JavaScriptSerializer()
Dim jsonData = jss.Deserialize(Of Object)(responseFromServer)
' List of key phrases to be returned
Dim phrases As New List(Of String)
For Each item As String In jsonData("documents")(0)("keyPhrases")
phrases.Add(item)
Next
Return phrases
My question is, what might I be doing wrong here, that I'm receiving messages that my document is exceeding the size limit of 10240 bytes, but it appears that the data that I POST is well under that limit.
As Assaf mentioned above, please make sure to specify UTF-8 encoding.

Accessing json object in vb.net after converting from XML

I convert XML to JSON like so:
Dim doc As XmlDocument
doc.LoadXml(arg_strXml)
Dim jsonObject As String = JsonConvert.SerializeXmlNode(doc)
Now I would like to be able to go into jsonObject to get values, like so
Dim mode As String = jsobObject.mode
Or more advanced example
Dim usersFirstName As String = jsonObject.people[1].firstname
Do I need to create a class which represents the XML structure or can I do it another way, even if I lose the intellisense, like so
Dim mode As String = jsonObject["mode"]
Seems like you first conversion was unnecessary,
you can access all values you need from xml with LINQ to XML and feature of vb.net XML Literals
Dim xmlString as String = "
<someobject>
<property1>Value 1</property1>
<property2>1001</property2>
</someobject>"
Dim xmlObject As XElement = XElement.Parse(xmlString)
Dim value1 As String = xmlObject.<property1>.First().Value
Dim value2 As String = xmlObject.<property2>.First().Value

Send RDLC in an email using vb.net

Okay my website generates thousands of PDFs using RDLCs but my problem is sometimes I want to email them but I don't want to attach a PDF to an email. So what I need is a way to generate the report then either convert it into text or html so I can send it as the body of an email.
Also I am using reportviewr version 11
Also I have tried exporting it as a .doc then trying to convert it to text and i have tried to export it to an excel document then tried to convert it and none of it works.
Dim warn() As Warning = Nothing
Dim streamids() As String = Nothing
Dim mimeType As String = String.Empty
Dim encoding As String = String.Empty
Dim extension As String = String.Empty
Dim bytes() As Byte
bytes = rv.LocalReport.Render("MHTML", Nothing, mimeType, encoding, extension, streamids, warn)
'Only one copy of the notice is needed
'If Not Directory.Exists(strFilePath) Then Directory.CreateDirectory(strFilePath)
Dim fs As New FileStream(strFilePath, FileMode.Create)
fs.Write(bytes, 0, bytes.Length)
fs.Close()
here is the code i'm using but it gives me an error : Specified argument was out of the range of valid values. Parameter name: format
Also i know this code works because I use the exact same thing to export the rdlc to a PDF
Ok so I solved my own problem with some Research about bytes.
Here is the code that I used to solve my problem.
What I did was exported the reportviewr as a word document and then converted all bytes to text. Then from that you end up with a whole bunch of gibberish but eventually you will find the text from your RDLC. So what I did was split the string up to where I was only left with the wording from my RDLC.
Review the code below:
Function GetRDLCText(ByVal rv As ReportViewer) As String
Dim warn() As Warning = Nothing
Dim streamids() As String = Nothing
Dim mimeType As String = String.Empty
Dim encoding As String = String.Empty
Dim extension As String = String.Empty
Dim bytes() As Byte
Dim msg() As String
bytes = rv.LocalReport.Render("WORD", Nothing, mimeType, encoding, extension, streamids, warn)
'Word is the only export that contains text from the rdlc
Dim content As String = System.Text.Encoding.Unicode.GetString(bytes)
msg = content.Split("Ù")
msg = msg(1).Split("Ѐ")
Return msg(0)
End Function
This solution is not for everyone, but it works for what I need it to do.

VB.NET ~ how does one navigate to a website and download the html then parse out code to only display input elements?

I have tried a few things like converting HTML to XML and then using an XML navigator to get input elements but I get lost whenever I start this process.
What I am trying to do is to navigate to a website which will be loaded using textbox1.text
Then download the html and parse out the input elements like . username, password, etc and place the element by type (id or name) into the richtextbox with the attribute beside the name.
Example.
Username id="username"
Password id="password"
Any clues or how to properly execute an HTML to XML conveter, reader, parser?
Thanks
It sounds like you just need a good HTML parsing library (instead of trying to use an XML parser). The HTML Agility Pack often fits this need. There are other options as well.
Somthing like below uses a streamreader to extract the source of the page into a string result
Dim uri As String = "https://www.yourUrl.com"
Dim request As HttpWebRequest = CType(WebRequest.Create(uri), HttpWebRequest)
Dim objRequest As HttpWebRequest = WebRequest.Create(uri)
Dim result As String
objRequest.Method = "GET"
Dim objResponse As HttpWebResponse = objRequest.GetResponse()
Dim sr As StreamReader
sr = New StreamReader(objResponse.GetResponseStream())
result = sr.ReadToEnd()
sr.Close
Then use regular expression (regex) to extra the attributes needed. for example something like this
Dim pattern As String = "(?<=Username id="")\w+"
Dim m0 As MatchCollection = Regex.Matches(result, pattern, RegexOptions.Singleline)
Dim m As Match
Dim k As Integer = 0
dim strUserID as String = ""
For Each m In m0
'extract the values for username id
strUserID = m0[k].Value;
k=k+1
Next
You'll need to change the pattern so it can pick up the other attributes you want to find, but this shouldn't be difficult

SSRS how to decode foreign language characters

I'm passing this string to a report: Economia e Administração
But the report displays the following: Economia e Administração
In the URL it gets encoded as:
Economia%20e%20Administra%C3%83%C2%83%C3%82%C2%A7%C3%83%C2%83%C3%82%C2%A3o%20
I tried using URLDecode, but it doesn't work.
Any ideas?
Thanks!
It looks like it's being converted into UTF-8 twice, ie, an encoded string is being encoded again. Is the original string being passed as Unicode or UTF-8 or something else?
CORRECTION: it's converted into UTF-8 three times!
Here's my solution...In parent report convert the string to a Byte array string and pass that to the child report:
Function GetStringBytes(ByVal theString As String) As String
Dim bytes() As Byte = System.Text.Encoding.UTF8.GetBytes(theString, 0, theString.Length)
Dim builder As New System.Text.StringBuilder
For Each i As Integer In bytes
builder.Append(i & "|")
Next i
Return builder.ToString().TrimEnd("|")
End Function
In the Child report pass the byte array string to the GetString function below to convert it back to the original string:
Function GetString(ByVal theBytes As String) As String
Dim byts() As Byte = New Byte(theBytes.Split("|").Length) {}
Dim count As Integer = 0
For Each i As String In theBytes.Split("|")
byts(count) = Convert.ToInt32(i)
count += 1
Next i
Return UTF8ByteArrayToString(byts)
End Function
Function UTF8ByteArrayToString(ByVal theChars As Byte()) As String
Dim aEncoding As System.Text.UTF8Encoding = New System.Text.UTF8Encoding()
Dim aConstructedString As String = aEncoding.GetString(theChars)
Return aConstructedString
End Function
Works perfect for me.