I've been reading the NodeMCU documentation and several closed issues about the change of SDK that previouly allowed to send multiple data streams (acting like a queued net.socket:send).
It seems a huge debate grew here (#730) and there (#993) or even here (#999). However, I did not find any convincing example of a webserver code that would allow me to read multiple html files (e.g. head.html and body.html) to display a page. Here's the example from TerryE that I tried to adapt, but with no success:
srv=net.createServer(net.TCP)
srv:listen(80,function(conn)
conn:on ("receive", function(sck, req)
local response = {}
local f = file.open("head.html","r")
if f ~= nil then
response[#response+1] = file.read()
file.close()
end
local f = file.open("body.html","r")
if f ~= nil then
response[#response+1] = file.read()
file.close()
end
local function sender (sck)
if #response>0 then sck:send(table.remove(response,1))
else sck:close()
end
end
sck:on("sent", sender)
sender(sck)
end )
end )
When connecting to the ESP8266, nothing loads and I get no error from the lua terminal.
For your information, head.html contains:
<html>
<head>
</head>
And body.html contains:
<body>
<h1>Hello World</h1>
</body>
</html>
I am very new to NodeMCU, please be tolerant.
Here is my solution without using tables, saving some memory:
function Sendfile(sck, filename, sentCallback)
if not file.open(filename, "r") then
sck:close()
return
end
local function sendChunk()
local line = file.read(512)
if line then
sck:send(line, sendChunk)
else
file.close()
collectgarbage()
if sentCallback then
sentCallback()
else
sck:close()
end
end
end
sendChunk()
end
srv = net.createServer(net.TCP)
srv:listen(80, function(conn)
conn:on("receive", function(sck, req)
sck:send("HTTP/1.1 200 OK\r\n" ..
"Server: NodeMCU on ESP8266\r\n" ..
"Content-Type: text/html; charset=UTF-8\r\n\r\n",
function()
Sendfile(sck, "head.html", function() Sendfile(sck, "body.html") end)
end)
end)
end)
And this is for serving single files:
function Sendfile(client, filename)
if file.open(filename, "r") then
local function sendChunk()
local line = file.read(512)
if line then
client:send(line, sendChunk)
else
file.close()
client:close()
collectgarbage()
end
end
client:send("HTTP/1.1 200 OK\r\n" ..
"Server: NodeMCU on ESP8266\r\n" ..
"Content-Type: text/html; charset=UTF-8\r\n\r\n", sendChunk)
else
client:send("HTTP/1.0 404 Not Found\r\n\r\nPage not found")
client:close()
end
end
srv = net.createServer(net.TCP)
srv:listen(80, function(conn)
conn:on ("receive", function(client, request)
local path = string.match(request, "GET /(.+) HTTP")
if path == "" then path = "index.htm" end
Sendfile(client, path)
end)
end)
Thank you for the reply. I actually added the header you mentioned, I didn't know that was necessary and I also removed the sck argument in the sender function. My first code was actually working, I don't know what was wrong last time.
Anyway, it helped me understanding what was happening: the following code seems to concatenate the response array, since the event sent of the socket calls back the sender function (sck:on("sent", sender))
sck:send(table.remove(response,1))
In fact, table.remove(array, 1) returns the first item of the array, and removes this item of the array. Calling this line multiple times has the effect to read through it, item by item.
For the sake of simplicity, here is the code of a simple webserver able to serve multiple files:
header = "HTTP/1.0 200 OK\r\nServer: NodeMCU on ESP8266\r\nContent-Type: text/html\r\n\r\n"
srv=net.createServer(net.TCP)
srv:listen(80,function(conn)
conn:on ("receive", function(sck, req)
local response = {header}
tgtfile = string.sub(req,string.find(req,"GET /") +5,string.find(req,"HTTP/") -2 )
if tgtfile == "" then tgtfile = "index.htm" end
local f = file.open(tgtfile,"r")
if f ~= nil then
response[#response+1] = file.read()
file.close()
else
response[#response+1] = "<html>"
response[#response+1] = tgtfile.." not Found - 404 error."
response[#response+1] = "<a href='index.htm'>Home</a>"
end
collectgarbage()
f = nil
tgtfile = nil
local function sender ()
if #response>0 then sck:send(table.remove(response,1))
else sck:close()
end
end
sck:on("sent", sender)
sender()
end)
end)
This example was taken from this instructables and fixed to work with the new SDK (which do not allow multiple :send anymore). Please let me know if this code has some issues.
I don't know what is the size limit of the files though. Nevertheless, I manage to append more than 2Ko to the response variable and send it at once without any issue.
Related
I have just started learning ruby and trying to learn extracting web data. I am using samplewebapp code given in a sites API documentation
On executing the test.rb file as given below it generates a token.
The token.json file contains the following
{"access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJhdWQiOiIzY2U1NjFhMC01ZmE5LTQ5NjUtYTQ2Mi01NmI2MzEzNjRlZGMiLCJqdGkiOiJmMjc0ZTgwM2FmZjVlMTFiMTAzNzIwMzAwNzRlNDRhZDE3ODg3ZjAzOGQ2ODk0OWIwNzVkMmQ5N2E2MjAwYzc5ZWRhNzU4MmQ3MTg1MTc1YiIsImlhdCI6MTY1NDU2OTc0MS41MjgxNTUsIm5iZiI6MTY1NDU2OTc0MS41MjgxNTcsImV4cCI6MTY1NDU3MzM0MS41MjgwMzksInN1YiI6IjE5NGFlNTU5LTU5OWMtNDQ2ZC1hOTRhLWM2MTIyYjZkMTA2ZiIsInNjb3BlcyI6W10sImNyZWRpdHNfcmVtYWluaW5nIjo0NzAwLCJyYXRlX2xpbWl0cyI6W3sicmF0ZSI6NSwiaW50ZXJ2YWwiOjYwfV19.XoN5A-H8Z7d8p_0e2rCcyV4yWO9MAF3TZlHk5VP5NjUEqtTXTVYKfKuzidYDs7K8ZzAtWzyt3aR5VSUG0_nw-uPd7Z3_Y75alQMqa2b5rHGn5JFWH7nuAriskr26WSCqAdj4cNjccPORryRr5sYKwpE4Y4kTG0IX_8frvwYxwp-8wFpLLj98K3axwN7CCpWdnPDSzuMqmH6tSpF0XhdYxB5LTVH0AyH7lN0S0_Lftq0b3sOLIvEaTfNkuRGNqwLkBYFkHFuPqwrKd8RJhC2W1QZhrmUw3eYnh-0iQABGk2V0skIqDlb6BbQ5GFX6MXgiXAc-h5Ndda7pZ5N5UCQU3g","expires_at":1654573341}
However it also generates a JSON parser error as under
/Users/mm/.rbenv/versions/3.0.4/lib/ruby/3.0.0/json/common.rb:216:in `parse': 809: unexpected token at '' (JSON::ParserError)
from /Users/mm/.rbenv/versions/3.0.4/lib/ruby/3.0.0/json/common.rb:216:in `parse'
from /Users/mm/Desktop/prokerala/client.rb:21:in `parseResponse'
from /Users/mm/Desktop/prokerala/client.rb:99:in `get'
I googled around for a while and some of the responses talk about the issues between double and single quotes.. I tried changing those also but it doesn't work.
I have also tried running it with system ruby - which gives a different error
Would appreciate very much if someone could explain what the error is / logic behind this.. Also I am not clear about the error message itself. The message talks about parse error 809:"unexpected token" on column no 809 of the token there is nothing.. Am i reading it wrong?
code used in test.rb file
client = ApiClient.new('YOUR CLIENT_ID', 'YOUR CLIENT_SECRET');
result = client.get('v2/astrology/thirumana-porutham/advanced', {
:girl_nakshatra => 4,
:girl_nakshatra_pada => 2,
:boy_nakshatra => 26,
:boy_nakshatra_pada => 3
})
puts JSON.pretty_generate(result)
code used in client.rb file
require 'net/http'
require 'json'
class ApiError < StandardError
end
class ApiClient
BASE_URL = "https://api.prokerala.com/"
# Make sure that the following file path is set to a location that is not publicly accessible
TOKEN_FILE = "./token.json"
def initialize(clientId, clientSecret)
# Instance variables
#clientId = clientId
#clientSecret = clientSecret
end
def parseResponse(response)
content = response.body
res = JSON.parse(content)
if res.key?('access_token')
return res
end
if res['status'] == "error"
raise ApiError, res['errors'].map {|e| e['detail']}.join("\n")
end
if res['status'] != "ok"
raise "HTTP request failed"
end
return res
end
def saveToken(token)
# Cache the token until it expires.
File.open(ApiClient::TOKEN_FILE,"w") do |f|
token = {
:access_token => token['access_token'],
:expires_at => Time.now.to_i + token['expires_in']
}
f.write(token.to_json)
end
end
def getTokenFromCache
if not File.file?(ApiClient::TOKEN_FILE)
return nil
end
begin
# Fetch the cached token, and return if not expired
text = File.read(ApiClient::TOKEN_FILE)
token = JSON.parse(text)
if token['expires_at'] < Time.now.to_i
return nil
end
return token['access_token']
rescue JSON::ParserError
return nil
end
end
def fetchNewToken
params = {
:grant_type => 'client_credentials',
:client_id => #clientId,
:client_secret => #clientSecret
}
res = Net::HTTP.post_form(URI(ApiClient::BASE_URL + 'token'), params)
token = parseResponse(res)
saveToken(token)
return token['access_token']
end
def get(endpoint, params)
# Try to fetch the access token from cache
token = getTokenFromCache
# If failed, request new token
token ||= fetchNewToken
uri = URI(ApiClient::BASE_URL + endpoint)
uri.query = URI.encode_www_form(params)
req = Net::HTTP::Get.new(uri.to_s, {'Authorization' => 'Bearer ' + token})
res = Net::HTTP.start(uri.hostname) do |http|
http.request(req)
end
return parseResponse(res)
end
end
running the test file using ruby 3.0.4 gives the a JSON parse error. Full error reproduced as below
/Users/mm/.rbenv/versions/3.0.4/lib/ruby/3.0.0/json/common.rb:216:in `parse': 809: unexpected token at '' (JSON::ParserError)
from /Users/mm/.rbenv/versions/3.0.4/lib/ruby/3.0.0/json/common.rb:216:in `parse'
from /Users/mm/Desktop/prokerala/client.rb:21:in `parseResponse'
from /Users/mm/Desktop/prokerala/client.rb:99:in `get'
from test.rb:11:in `<main>'
I executed the above code and from this, we are getting
#<Net::HTTPMovedPermanently 301 Moved Permanently readbody=true> as response
and response body as empty string '', which results in error in parsing the body.
Since, API endpoint is secure with https, we need set use_ssl: true before starting the session. Try below code and it will fix the issue.
res = Net::HTTP.start(uri.hostname, use_ssl: true) do |http|
http.request(req)
end
I am new to lua and i am trying to make a function that receives documents and outputs a table but I am getting the above error. Why??
io.write("How many documents are we evaluating? \nInput: ")
local total_documents=io.read("*n")
io.read()
local docTable = {}
inputDocument()
function inputDocument()
local input
local file
local inputFile = {filename = nil, contents = nil, wordcount = nil}
repeat
io.write("Please enter document (filename.extension): ")
input = io.read()
file =io.open(input)
if file == nil then
print("File does not exist try again")
end
until(file ~=nil)
inputFile.filename = input
return inputFile
end
You need to define inputDocument before using it:
function inputDocument()
...
end
io.write("How many documents are we evaluating? \nInput: ")
...
inputDocument()
I am using http.get() to get a JSON from an API I am using, but it's not getting the data. I have the suspicion that this JSON is too big for the NodeMCU. I only need the information in the subpart "stats:". Is it possible to only http.get() that part of the JSON?
EDIT:
This is my code
function getstats()
http.get("https://api.aeon-pool.com/v1/stats_address?address=WmsGUrXTR7sgKmHEqRNLgPLndWKSvjFXcd4soHnaxVjY3aBWW4kncTrRcBJJgUkeGwcHfzuZABk6XK6qAp8VmSci2AyGHcUit", nil, function(code, pool)
if (code < 0) then
print("can't get stats")
else
h = cjson.decode(pool)
hashrate = h[1]["hashrate"]
print(hashrate)
dofile('update_display.lua')
end
end)
end
I also have another function getting data from another api above getstats()
function getaeonrate()
http.get("https://api.coinmarketcap.com/v1/ticker/aeon/?convert=EUR", nil, function(code, dataaeon)
if (code < 0) then
print("can't get aeon")
else
-- Decode JSON data
m = cjson.decode(dataaeon)
-- Extract AEON/EUR price from decoded JSON
aeonrate = string.format("%f", m[1]["price_eur"]);
aeonchange = "24h " .. m[1]["percent_change_24h"] .. "% 7d " .. m[1]["percent_change_7d"] .. "%"
dofile('update_display.lua')
end
end)
end
But now the weird thing is, when I want to access 'pool' from getstats() I get the json data from getaeonrate(). So "hashrate" isn't even in the json because I am getting the json from another function.
I tried making a new project only with getstats() and that doesn't work at all I always get errors like this
HTTP client: Disconnected with error: -9
HTTP client: Connection timeout
HTTP client: Connection timeout
Yesterday I thought that the response was too big from api.aeon-pool.com, I if you look at the json in your webbrowser you can see that the top entry is 'stats:' and I only need that, none of the other stuff. So If the request is to big It would be nice to only http.get() that part of the json, hence my original question. At the moment I am not even sure what is not working correctly, I read that the nodemcu firmware generally had problems with http.get() and that it didn't work correctly for a long time, but getting data from api.coinmarketcap.com works fine in the original project.
The problems with the HTTP module are with near certainty related to https://github.com/nodemcu/nodemcu-firmware/issues/1707 (SSL and HTTP are problematic).
Therefore, I tried with the more bare-bone TLS module on the current master branch. This means you need to manually parse the HTTP response including all headers looking for the JSON content. Besides, you seem to be on an older NodeMCU version as you're still using CJSON - I used SJSON below:
Current NodeMCU master branch
function getstats()
buffer = nil
counter = 0
local srv = tls.createConnection()
srv:on("receive", function(sck, payload)
print("[stats] received data, " .. string.len(payload))
if buffer == nil then
buffer = payload
else
buffer = buffer .. payload
end
counter = counter + 1
-- not getting HTTP content-length header back -> poor man's checking for complete response
if counter == 2 then
print("[stats] done, processing payload")
local beginJsonString = buffer:find("{")
local jsonString = buffer:sub(beginJsonString)
local hashrate = sjson.decode(jsonString)["stats"]["hashrate"]
print("[stats] hashrate from aeon-pool.com: " .. hashrate)
end
end)
srv:on("connection", function(sck, c)
sck:send("GET /v1/stats_address?address=WmsGUrXTR7sgKmHEqRNLgPLndWKSvjFXcd4soHnaxVjY3aBWW4kncTrRcBJJgUkeGwcHfzuZABk6XK6qAp8VmSci2AyGHcUit HTTP/1.1\r\nHost: api.aeon-pool.com\r\nConnection: close\r\nAccept: */*\r\n\r\n")
end)
srv:connect(443, "api.aeon-pool.com")
end
Note that the receive event is fired for every network frame: https://nodemcu.readthedocs.io/en/latest/en/modules/net/#netsocketon
NodeMCU fails to establish a connection to api.coinmarketcap.com due to a TLS handshake failure. Not sure why that is. Otherwise your getaeonrate() could be implemented likewise.
Frozen 1.5.4 branch
With the old branch the net module can connect to coinmarketcap.com.
function getaeonrate()
local srv = net.createConnection(net.TCP, 1)
srv:on("receive", function(sck, payload)
print("[aeon rate] received data, " .. string.len(payload))
local beginJsonString = payload:find("%[")
local jsonString = payload:sub(beginJsonString)
local json = cjson.decode(jsonString)
local aeonrate = string.format("%f", json[1]["price_eur"]);
local aeonchange = "24h " .. json[1]["percent_change_24h"] .. "% 7d " .. json[1]["percent_change_7d"] .. "%"
print("[aeon rate] aeonrate from coinmarketcap.com: " .. aeonrate)
print("[aeon rate] aeonchange from coinmarketcap.com: " .. aeonchange)
end)
srv:on("connection", function(sck, c)
sck:send("GET /v1/ticker/aeon/?convert=EUR HTTP/1.1\r\nHost: api.coinmarketcap.com\r\nConnection: close\r\nAccept: */*\r\n\r\n")
end)
srv:connect(443, "api.coinmarketcap.com")
end
Conclusion
The HTTP module and TLS seem a no-go for your APIs due to a bug in the firmware (1707).
The net/TLS module of the current master branch manages to connect to api.aeon-pool.com but not to api.coinmarketcap.com.
With the old and frozen 1.5.4 branch it's exactly the other way around.
There may (also) be issues with cipher suits that don't match between the firmware and the API provider(s).
-> :( no fun like that
How to post data to my own server. I can fetch data from it but can't add data .Code for fetch is below:
local function networkListener( event )
if ( event.isError ) then
print( "Network error!")
else
local json=event.response
local length=string.len(json)
json=string.sub(json, 50, (length-1));
jsonTable=JSON.encode(json)
local t = jsonTable
print(jsonTable)
-- Go through the array in a loop
for key in pairs(t) do
-- Here you can do whatever you like with the values
print(t[key]["AuthorID"])
print(t[key]["AuthorName"])
--print(t[key]["returnvalue3"])
end
end
end
local remoteFeed="http://www.xtremeesolutions.com/xesapps/webservice/readauthors.php"
network.request(remoteFeed, "GET", networkListener)
--And trying code to post data is
local function postData(e)
if (e.isError)then
print("Error ");
else
print("Error ".. e.response);
end
end
local params = {
body = query
}
network.request(remoteFeed, "POST",postData,params))
Above code not giving any error but also not inserting values.Please suggest me to solve this.
I think you should use "POST" instead of "GET", see the code below:
...
local remoteFeed="http://www.xtremeesolutions.com/xesapps/webservice/readauthors.php"
network.request(remoteFeed, "POST", networkListener, params)
Maybe you should also check this link http://docs.coronalabs.com/api/library/network/request.html at section: HTTP POST with custom headers
EDITED
the loop for handle post value should be:
if ( event.isError ) then
print( "Network error!")
else
...
for key in pairs(t) do
postData = t[key]["AuthorID"] .. "=" .. t[key]["AuthorName"]
end
local params = {}
params.body = postData
local remoteFeed="http://www.xtremeesolutions.com/xesapps/webservice/readauthors.php"
network.request(remoteFeed, "POST", networkListener, params)
end
those lines should be togather in else scope.
I'm trying to write a function that will read sound and music states before starting my application.
The problem is: The first time it will run, there will be no data recorded.
First, I tried the suggested JSON function from here and I got this error:
Attempt to call global 'saveTable' (a nil value)
Is there a way to test if the file exists?
Then, I tried this one:
-- THIS function is just to try to find the file.
-- Load Configurations
function doesFileExist( fname, path )
local results = false
local filePath = system.pathForFile( fname, path )
--filePath will be 'nil' if file doesn,t exist and the path is "system.ResourceDirectory"
if ( filePath ) then
filePath = io.open( filePath, "r" )
end
if ( filePath ) then
print( "File found: " .. fname )
--clean up file handles
filePath:close()
results = true
else
print( "File does not exist: " .. fname )
end
return results
end
local fexist= doesFileExist("optionsTable.json","")
if (fexist == false) then
print (" optionsTable = nil")
optionsTable = {}
optionsTable.soundOn = true
optionsTable.musicOn = true
saveTable(optionsTable, "optionsTable.json") <<<--- ERROR HERE
print (" optionsTable Created")
end
The weird thing is that I'm getting an error at the saveTable(optionsTable,"optionsTable.json"). I just can't understand why.
If you have a working peace of code that handles the first time situation it will be enough to me. Thanks.
here's some code to check if the file exist you have to try and open the file first to know if it exist
function fileExists(fileName, base)
assert(fileName, "fileName is missing")
local base = base or system.ResourceDirectory
local filePath = system.pathForFile( fileName, base )
local exists = false
if (filePath) then -- file may exist wont know until you open it
local fileHandle = io.open( filePath, "r" )
if (fileHandle) then -- nil if no file found
exists = true
io.close(fileHandle)
end
end
return(exists)
end
and for usage
if fileExists("myGame.lua") then
-- do something wonderful
end
you can refer to this link for details