I'am currenty using code :
Private Function getHtmlres(ByVal response As HttpWebResponse) As String
Dim myWebSource As New StreamReader(response.GetResponseStream())
Dim myPageSource As String = String.Empty
myPageSource = myWebSource.ReadToEnd()
Return myPageSource
End Function
But the problem is that I'm downloading audio/mpeg object. When I see the
MsgBox(getHtmlres(myHttpWebResponse1))
It shows :
ID3<<<SQUARE>>>
. When I do
System.IO.File.WriteAllText("D:\sdg.txt",getHtmlres(myHttpWebResponse1))
it writes 0 bytes.
Related
I get an error when executing the following code:
Public Function GetNumberInfo(ByVal pPhoneNumber As String) As PhoneInformation
Dim url As String = "https://api.phonevalidator.com/api/v3/phonesearch?apikey=[My_API_Key]&phone=" & pPhoneNumber & "&type=fake,basic"
DebugLog("App:Registration (RegistrationMOD.vb/GetNumberInfo) - DEBUG URL string: " & url)
Dim request = CType(WebRequest.Create(url), HttpWebRequest)
Dim myReader As StreamReader
request.Method = "GET"
request.Accept = "application/json"
request.ContentType = "application/json; charset=utf-8;"
' The error occurs on the execution of this line.
Dim response As HttpWebResponse = CType(request.GetResponse(), HttpWebResponse)
myReader = New System.IO.StreamReader(response.GetResponseStream)
Dim responseString As String = ""
responseString = myReader.ReadToEnd
Dim j As JavaScriptSerializer = New JavaScriptSerializer()
Dim MyPhone As PhoneInformation = j.Deserialize(Of PhoneInformation)(responseString)
Return MyPhone
End Function
The strange this is, this EXACT same code executes fine in a test console app I created to test my conversion from c# code to vb code. I log all errors generated by the application and the following log entry details are what's in the log:
Debug log entries:
App:Registration/CreateAccount - An error occurred executing CheckPhone, the error is: The underlying connection was closed: An unexpected error occurred on a receive.
App:Registration/CreateAccount - Phone number submitted: 7029270000
Trace: System.Net.WebException(0x80131509): The underlying connection was closed: An unexpected error occurred on a receive.
at Microsoft.VisualBasic.CompilerServices.Symbols.Container.InvokeMethod(Method TargetProcedure, Object[] Arguments, Boolean[] CopyBack, BindingFlags Flags)
at Microsoft.VisualBasic.CompilerServices.NewLateBinding.ObjectLateGet(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack)
at Microsoft.VisualBasic.CompilerServices.NewLateBinding.LateGet(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack)
at Registration.RegistrationMod.GetNumberInfo(String pPhoneNumber) in C:\Users\Presc\Documents\Visual Studio 2013\Projects\Registration\Registration\RegistrationMod.vb:line 20
at Registration.CreateAccount.CheckPhone(String pPhoneNumber) in C:\Users\Presc\Documents\Visual Studio 2013\Projects\Registration\Registration\CreateAccount.aspx.vb:line 241
Line:20 Filename: RegistrationMod.vb
Line:241 Filename: CreateAccount.aspx.vb
This error occurrs on the execution of the line:
Dim response As HttpWebResponse = CType(request.GetResponse(), HttpWebResponse)
The debug log entry generated for the url string generated by the app to be submitted to the API is:
App:Registration (RegistrationMOD.vb/GetNumberInfo) - DEBUG URL string: https://api.phonevalidator.com/api/v3/phonesearch?apikey=[My_API_Key]&phone=7029270000&type=fake,basic
I don't understand how it could work in the console app and not in this app. The console app code is:
Imports System.IO
Imports System.Net
Imports Nancy.Json
Module TestCodeMOD
Public Function GetNumberInfo(ByVal pPhoneNumber As String) As PhoneInformation
Dim url As String = "https://api.phonevalidator.com/api/v3/phonesearch?apikey=[My_API_Key]&phone=" & pPhoneNumber & "&type=fake,basic"
Dim request = CType(WebRequest.Create(url), HttpWebRequest)
Dim myReader As StreamReader
request.Method = "GET"
request.Accept = "application/json"
request.ContentType = "application/json; charset=utf-8;"
Dim response As HttpWebResponse = CType(request.GetResponse(), HttpWebResponse)
myReader = New System.IO.StreamReader(response.GetResponseStream)
Dim responseString As String = ""
responseString = myReader.ReadToEnd
Dim j As JavaScriptSerializer = New JavaScriptSerializer()
Dim MyPhone As PhoneInformation = j.Deserialize(Of PhoneInformation)(responseString)
Return MyPhone
End Function
End Module
I execute the function like so:
Sub Main()
Dim MyPhone As PhoneInformation
MyPhone = GetNumberInfo("7029270000")
End Sub
The console app executes without any errors and populates the PhoneInformation class instance with the expected data. I'm at a loss as to what's going on and I'm open to suggestions.
I am struggling to fill a DataGridView from a JSON that I get through a webrequest to SOLR.
JSON Example:
{
"response":{"numFound":6,"start":1,"docs":[
{
"PRODUCTNAME":"Office Chair",
"CURRENCYCODE":"EUR",
"CLIENTCODE":"Northwind Inc",
"LANGUAGECODE":"ENG",
"KEYWORDS":"spins, adjust, castors"}]
}}
The below will work and get one token and put it in a label.
Code:
Private Sub SOLR()
Label2.Text = Nothing
Try
Dim fr As WebRequest
Dim targetURI As New Uri("LinkToJson")
fr = DirectCast(WebRequest.Create(targetURI), WebRequest)
fr.Credentials = New NetworkCredential("admin", "admin")
If (fr.GetResponse().ContentLength > 2) Then
Dim str As New StreamReader(fr.GetResponse().GetResponseStream())
Dim streamText As String = str.ReadToEnd()
Dim myJObject = JObject.Parse(streamText)
Label2.Text = myJObject.SelectToken("response.docs[0].KEYWORDS")
Label3.Text = streamText
End If
Catch ex As WebException
MessageBox.Show(ex.ToString())
End Try
End Sub
I tried to deserialize it, but I get an error for the below:
Dim table As DataTable = JsonConvert.DeserializeObject(Of DataTable)(streamText)
DataGridView1.DataSource = myJObject
Newtonsoft.Json.JsonSerializationException: 'Unexpected JSON token when reading DataTable. Expected StartArray, got StartObject. Path '', line 1, position 1.'
You need to pass the "docs" array to the DeserializeObject function in order to load the JSON data into DataTable.
Dim myJObject = JObject.Parse(streamText)
Dim arr = myJObject("response")("docs")
Dim table = JsonConvert.DeserializeObject(Of DataTable)(arr.ToString())
I'm using code below to get data in JSON format using Newtonsoft.Json:
my code to create the JSON is
Dim sb As New StringBuilder()
Dim sw As New StringWriter(sb)
Dim writer As JsonWriter = New JsonTextWriter(sw)
writer.WriteStartArray()
For Each elements As JToken In result
If i = 0 Then
i += 1
Continue For
End If
For Each aaa As JToken In elements
writer.WriteStartObject()
Dim pmid = aaa("uid").Value(Of String)
Dim issn = aaa("issn").Value(Of String)
Dim essn = aaa("essn").Value(Of String)
Dim source = aaa("source").Value(Of String)
Dim sortpubdate = aaa("sortpubdate").Value(Of String)
writer.WritePropertyName("pmid")
writer.WriteValue(pmid)
writer.WritePropertyName("journal")
writer.WriteValue(source)
writer.WritePropertyName("issn")
writer.WriteValue(issn)
writer.WritePropertyName("essn")
writer.WriteValue(essn)
writer.WritePropertyName("sortpubdate")
writer.WriteValue(sortpubdate)
writer.WritePropertyName("pubYear")
writer.WriteEndObject()
Next
Next
writer.Close()
Return sb.ToString
and the output code is
public HttpResponseMessage Get()
{
string yourJson = GetJsonFromSomewhere();
var response = this.Request.CreateResponse(HttpStatusCode.OK);
response.Content = new StringContent(yourJson, Encoding.UTF8, "application/json");
return response;
}
My current output is.
[{"pmid":"29241721","issn":"0749-3797","essn":"1873-2607","journal":"Am J Prev Med","pubYear":"2018","ImpactFactor":" 4.127"},{"pmid":"28987617","issn":"0166-4328","essn":"1872-7549","journal":"Behav Brain Res","pubYear":"2018","ImpactFactor":" 3.173"},
Is there a way I can indent my output?
[{
"pmid": "29241721",
"issn": "0749-3797",
"essn": "1873-2607",
"journal": "Am J Prev Med",
"pubYear": "2018",
"ImpactFactor": " 4.127"
}, {
Is there a way I can do it like below?
this.SerializerSettings.Formatting = Formatting.Indented;
The client side is just a simple browser URL that get a response
You should indent your object in serialization time and then use it everywhere you want. If GetJsonFormatSomewhere() is your serialization method you should seralize your object in that method below:
var serializedObject = JsonConvert.SerializeObject(data, Formatting.Indented);
and then pass indented object to StringContent method.
Note: it's better to indent your output in client side, that way your packet size will stay small and it's performance friendly.
I got what I needed with
Dim writer As JsonWriter = New JsonTextWriter(sw)
writer.Formatting = Newtonsoft.Json.Formatting.Indented
I'm new to VB.NET and am working on a simple program that will serialize two variables into JSON and POST the data to a web server. The data is being received by the Python server but it is giving an error when trying to deserialize the data.
The inputs are:
tester_id = 2
operation = "P"
When serialized this looks like:
{
"tester_id": 2,
"operation": "P"
}
Our server is giving an error and has the following log.
[Sat Aug 19 13:46:53.485257 2017] [:error] [pid 17352] <QueryDict: {u'{\\r\\n "tester_id": 2,\\r\\n "operation": "P"\\r\\n}': [u'']}>
This suggests that it is receiving a key of:
{u'{\\r\\n "tester_id": 2,\\r\\n "operation": "P"\\r\\n}
With a value of:
[u'']
This isn't correct and I don't understand why it is being received like this, any help is greatly appreciated! See below for the VB.NET code.
Class:
Public Class JSON_get_sensor_id_POST
Public Property tester_id() As Integer
Get
Return m_tester_id
End Get
Set(ByVal value As Integer)
m_tester_id = value
End Set
End Property
Private m_tester_id As Integer
Public Property operation() As String
Get
Return m_operation
End Get
Set(ByVal value As String)
m_operation = value
End Set
End Property
Private m_operation As String
End Class
Call function:
Dim set_tester_id As Integer = 1
Dim set_operation As String = "P"
Dim manuf_url As String = "https://XYZ...."
Dim JSON_to_send As New JSON_get_sensor_id_POST
JSON_to_send.tester_id = set_tester_id
JSON_to_send.operation = set_operation
Dim postData = JsonConvert.SerializeObject(JSON_to_send, Formatting.Indented)
Dim return_object = POST_to_Server(manuf_url, postData)
Upload Function:
Private Function POST_to_Server(ByVal post_url As String, ByVal JSON_to_post As Object)
Dim user_login As String = "blah#blah.com"
Dim user_pass As String = "blah"
Dim myCache As New CredentialCache()
myCache.Add(New Uri(post_url), "Basic", New NetworkCredential(user_login, user_pass))
' Create a request using a URL that can receive a post.
Dim request As WebRequest = WebRequest.Create(post_url)
' Set the Method property of the request to POST.
request.Credentials = myCache
request.Method = "POST"
request.ContentType = "application/json"
' Create POST data and convert it to a byte array.
Dim byteArray As Byte() = Encoding.Default.GetBytes(JSON_to_post)
' Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length
' Get the request stream.
Dim dataStream As Stream = request.GetRequestStream()
' Write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length)
' Close the Stream object.
dataStream.Close()
' Get the response.
Dim response As WebResponse = request.GetResponse()
' Display the status.
Console.WriteLine(CType(response, HttpWebResponse).StatusDescription)
' Get the stream containing content returned by the server.
dataStream = response.GetResponseStream()
' Open the stream using a StreamReader for easy access.
Dim reader As New StreamReader(dataStream)
' Read the content.
Dim responseFromServer As String = reader.ReadToEnd()
' Display the content.
DebugMessage(responseFromServer)
' Clean up the streams.
reader.Close()
dataStream.Close()
response.Close()
Dim myObject = JsonConvert.DeserializeObject(Of JSON_sensor_id_request_return)(responseFromServer)
Return myObject
End Function
\r and \n are escape sequences that are not visible when a string is displayed. Instead of formatting your data like this:
{
"tester_id": 2,
"operation": "P"
}
try formatting it something like this:
{"tester_id": 2,"operation": "p"}
I'm using Sharefile API that sends HTTP requests and gets their respective answers. They are made through URL's and use always the same function. Here it is.
Private Function InvokeShareFileOperation(ByVal requestUrl As String) As JObject
Dim request As HttpWebRequest = WebRequest.Create(requestUrl)
Dim response As HttpWebResponse = request.GetResponse()
Dim reader As StreamReader = New StreamReader(response.GetResponseStream())
Dim json As String = reader.ReadToEnd()
response.Close()
Return JObject.Parse(json)
End Function
As some operations are a bit long, I need to somehow track their progress while they are underway and don't know how to do it. Later I'm intending to use that progress and create a progress bar from it.
(EDIT)
By the way, it's the second code line (below) that takes most time, that is the operation to track.
Dim response As HttpWebResponse = request.GetResponse()
EDIT: I don't think you are going to be able to measure progress in any accurate way here as the bulk of the operation seems to be reliant on the remote server processing the request. GetResponse() handles setting up the DNS, connecting, sending and waiting for the remote server and this is all out of your hands. Reading the response stream is only measurable if the content-length header is returned. Personally I would show progress as 20% initially, 60% when GetResponse returns, and then the last 40% could be shown incrementally if you have the content length before downloading, or done in one go once you have finished reading the response stream.
As its a web request you can find out the content length first and then read the response stream using a buffer instead of ReadToEnd(). This allows you to calculate the progress and fire off notifications while downloading the response.
Dim request As HttpWebRequest = WebRequest.Create(requestUrl)
Using response As HttpWebResponse = request.GetResponse()
Dim contentLength As Long = response.ContentLength
Dim bytesReceived As Long
Dim bufferLength as Integer = 8192
Dim buffer(bufferLength) As Char
Dim sb As New StringBuilder
Using reader As StreamReader = New StreamReader(response.GetResponseStream())
Do
Dim bufferedCount As Integer = reader.Read(buffer, 0, bufferLength)
sb.Append(buffer, 0, bufferedCount)
bytesReceived += bufferedCount
Console.WriteLine(bytesReceived / contentLength * 100 & "%")
Loop While bytesReceived < contentLength
End Using
Return JObject.Parse(sb.ToString)
End Using
Obviously you can substitute the Console.WriteLine with a progress update function or a call to a SignalR hub to update a web page, and you can experiment with the buffer size to see what works best for you.
First we must find out what's slowing down. Request isn't send until GetResponse() is called, so processing by server can take some time. Downloading can also take some time. If response is small (relative to connection speed), you can't do much (you can if server is yours, but we'll focus on client) because you can't get progress from server. If response is large, and you want to track downloading, you can only do it if you have Content-Length header. And to get only headers, server must support HEAD request method. So here is code :
Imports System
Imports System.Net
Imports System.IO
Imports System.Text
Imports System.Threading
Imports Microsoft.VisualBasic
Public Class Form1
Private Function InvokeShareFileOperation(ByVal requestUrl As String) As JObject
HTTPWebRequest_GetResponse.Main(requestUrl)
ProgressBar1.Value = 0
Dim result As String
Do
Try
ProgressBar1.Value = HTTPWebRequest_GetResponse.progress
Catch ex As ArgumentOutOfRangeException
ProgressBar1.Style = ProgressBarStyle.Marquee
End Try
If HTTPWebRequest_GetResponse.done = True Then
result = HTTPWebRequest_GetResponse.response
ProgressBar1.Style = ProgressBarStyle.Continuous
ProgressBar1.Value=100
Debug.WriteLine(result)
Return JObject.Parse(result)
Exit Do
End If
Loop
End Function
End Class
Public Class RequestState
' This class stores the State of the request.
Private BUFFER_SIZE As Integer = 1024
Public requestData As StringBuilder
Public BufferRead() As Byte
Public request As HttpWebRequest
Public response As HttpWebResponse
Public streamResponse As Stream
Public Sub New()
BufferRead = New Byte(BUFFER_SIZE) {}
requestData = New StringBuilder("")
request = Nothing
streamResponse = Nothing
End Sub 'New
End Class 'RequestState
Class HTTPWebRequest_GetResponse
Private BUFFER_SIZE As Integer = 1024
Public Shared response As String
Public Shared done As Boolean = False
Public Shared length As Long = 1
Public Shared progress As Integer
Public Shared myHttpWebRequest As HttpWebRequest
Public Shared myRequestState As New RequestState()
Shared Sub Main(url As String)
Try
Dim headRequest As HttpWebRequest = WebRequest.Create(url)
headRequest.Method = "HEAD"
Dim headResponse As HttpWebResponse = headRequest.GetResponse
length = headResponse.ContentLength
Debug.WriteLine(length)
headResponse.Close()
' Create a HttpWebrequest object to the desired URL.
myHttpWebRequest = WebRequest.Create(url)
' Create an instance of the RequestState and assign the previous myHttpWebRequest
' object to its request field.
myRequestState.request = myHttpWebRequest
'Dim myResponse As New HTTPWebRequest_GetResponse()
' Start the asynchronous request.
Dim result As IAsyncResult = CType(myHttpWebRequest.BeginGetResponse(New AsyncCallback(AddressOf RespCallback), myRequestState), IAsyncResult)
Catch e As WebException
Debug.WriteLine("Main Exception raised!")
Debug.WriteLine("Message: " + e.Message)
Debug.WriteLine("Status: " + e.Status)
Catch e As Exception
Debug.WriteLine("Main Exception raised!")
Debug.WriteLine("Source : " + e.Source)
Debug.WriteLine("Message : " + e.Message)
End Try
End Sub 'Main
Private Shared Sub RespCallback(asynchronousResult As IAsyncResult)
Debug.WriteLine("RespCallBack entered")
Try
' State of request is asynchronous.
Dim myRequestState As RequestState = CType(asynchronousResult.AsyncState, RequestState)
Dim myHttpWebRequest As HttpWebRequest = myRequestState.request
myRequestState.response = CType(myHttpWebRequest.EndGetResponse(asynchronousResult), HttpWebResponse)
' Read the response into a Stream object.
Dim responseStream As Stream = myRequestState.response.GetResponseStream()
myRequestState.streamResponse = responseStream
' Begin the Reading of the contents of the HTML page.
Dim asynchronousInputRead As IAsyncResult = responseStream.BeginRead(myRequestState.BufferRead, 0, 1024, New AsyncCallback(AddressOf ReadCallBack), myRequestState)
Return
Catch e As WebException
Debug.WriteLine("RespCallback Exception raised!")
Debug.WriteLine("Message: " + e.Message)
Debug.WriteLine("Status: " + e.Status)
Catch e As Exception
Debug.WriteLine("RespCallback Exception raised!")
Debug.WriteLine("Source : " + e.Source)
Debug.WriteLine("Message : " + e.Message)
End Try
End Sub 'RespCallback
Private Shared Sub ReadCallBack(asyncResult As IAsyncResult)
Debug.WriteLine("ReadCallBack entered")
Try
Dim myRequestState As RequestState = CType(asyncResult.AsyncState, RequestState)
Dim responseStream As Stream = myRequestState.streamResponse
Dim read As Integer = responseStream.EndRead(asyncResult)
' Read the HTML page.
If read > 0 Then
myRequestState.requestData.Append(Encoding.ASCII.GetString(myRequestState.BufferRead, 0, read))
If length = -1 Or length = 0 Then
progress = -1
Else
progress = myRequestState.BufferRead.Length * 100 / length
Debug.WriteLine(progress)
End If
Dim asynchronousResult As IAsyncResult = responseStream.BeginRead(myRequestState.BufferRead, 0, 1024, New AsyncCallback(AddressOf ReadCallBack), myRequestState)
Else
If myRequestState.BufferRead.Length > 1 Then
Dim fullResponse As String = myRequestState.requestData.ToString
response = fullResponse.Substring(0, fullResponse.IndexOf("</body>")).Substring(fullResponse.IndexOf(">", fullResponse.IndexOf("<body")) + 2) 'Returns only body
' Release the HttpWebResponse resource.
myRequestState.response.Close()
done = True
Debug.WriteLine(done)
End If
responseStream.Close()
End If
Catch e As WebException
Debug.WriteLine("ReadCallBack Exception raised!")
Debug.WriteLine("Message: " + e.Message)
Debug.WriteLine("Status: " + e.Status)
Catch e As Exception
Debug.WriteLine("ReadCallBack Exception raised!")
Debug.WriteLine("Source : " + e.Source)
Debug.WriteLine("Message : " + e.Message)
End Try
End Sub 'ReadCallBack
End Class 'HttpWebRequest_BeginGetResponse
I took code from http://msdn.microsoft.com/en-us/library/debx8sh9(v=vs.110).aspx and changed it.
EDIT: Code now returns only body and response is closed.
EDIT2: As #Geezer68 said, it's not 100% accurate, but it's OK for showing progress to user.
I'm pretty sure what you want is reader.BaseStream.Length so you can know the length before reading. (At least I did, so I tried) But it threw a NotSupportedException with the message This stream does not support seek operations. So I googled StreamReader + This stream... and found this SO link:
Error “This stream does not support seek operations” in C#
So the short answer is: It is not possible.
Maybe a simple stopwatch is a way to start with ?
Dim timer As System.Diagnostics.Stopwatch = New Stopwatch()
Dim request As HttpWebRequest = WebRequest.Create(requestUrl)
timer.Start()
Dim response As HttpWebResponse = request.GetResponse()
timer.Stop()
Dim reader As StreamReader = New StreamReader(response.GetResponseStream())
Dim json As String = reader.ReadToEnd()
response.Close()
Label1.Text = "Secs:" & timer.Elapsed.ToString()
Here is a link to Microsoft Example
where you set the buffer size and a call back on the response object
https://msdn.microsoft.com/en-us/library/86wf6409%28v=vs.110%29.aspx