I am new to programming and am working on pushing real time data from a PLC to a web page either by deploying HTML 5 on the WAGO or a Modbus driver wrapper. I honestly have tried to research but don't know where to start. it will be a closed private network with little to no influence from the outside web. I am simply looking to display a single piece of live information for proof of concept. basically I'm trying to custom design a Groov program.
You might want to look into using OPC. Kepware & SoftwareToolbox are just 2 of many vendors that offer tools to help you get your data the way you want it.
There is an existing tool to do what you want, but I am under the impression you have to write one from scratch. The existing tool is http://www.softwaretoolbox.com/cogentdatahub/ if you are interested in looking at it for ideas.
I've been able to interface with PLC using python and modbusTCP and an Raspberry pi as the webserver. Python is a quick and easy to learn language. Websockets are the HTML5 component best used for realtime data.
simple connect code (after you install everything):
from pymodbus.client.sync import ModbusTcpClient as ModbusClient
from time import sleep
client = ModbusClient('ip_address_of_modbus_IO')
if(client.connect()):
print(client.read_discrete_inputs(200,1).bits[0])
client.write_coil(0,True)
sleep(100)
client.write_coil(2,True)
found here:
http://simplyautomationized.blogspot.com/2013/09/home-automation-project-2-rpi-light.html
Can create a websocket broadcast server using an example here:
http://simplyautomationized.blogspot.com/2015/09/raspberry-pi-create-websocket-api.html
Fortunately you can not push data to a browser.
The Internet would become an even greater mess if you could.
To solve this, have your webpage contain a timer, written in JavaScript.
Every say 1 sec. it does an AJAX request (e.g. use jQuery implementation) to the server, which then delivers (almost) realtime data.
The webpage then displays that in some DOM element, e.g. an empty DIV.
So it's the browser polling your server.
#BlueDog
The data is "almost" realtime because sampling once a second gives a delay of at least one second. In the ideal case, as soon as data changes, it would be pushed to the browser. Unfortunately the browser has no way of knowing that anything changed, so the best it can do is frequently "ask" for updates (polling).
How much the delay is depends on your poll frequency. If it's once per second one has to add the delays for transmission of the page request and the reply of the server. The transmission time depends on your network (which may be the Internet with all uncertainty involved). If the backbones involved have enough capacity I expect overall delay to be between 1 and 1.5 seconds. With a dedicated network and even more frequent polling, I expect that 0.5 seconds should be possible. These are however estimated averages. If I request a page over the Internet and my provider (again) has a problem, it may be hours before I receive what I want. Also things like virus scanners and OS updates may spoil your game.
So, practically: with a good broadband connection, a stable browser and the right process priorities it should be possible to get below 1 second overall delay (incl. poll time interval) for 95% of the time. Be prepared to reboot the client every few days. Most browsers leak memory and most OS'es do so too.
Related
What is the term for a webhook that makes another set of webhooks?
A recent kludge (Unbounce, Zapier, Twilio) only allows me one webhook (i.e. custom one from Zapier)
https://zapier.com/zapbook/twilio/unbounce/
Eventually, however, I will need to generate multiple actions on different (customzed) platforms, so I was hoping to have a hook that goes to one place and then from there, vector off hooks to my other beta site (including Zapier).
Any suggestions would be greatly appreciated.
I note your question was about unbounce, but included Zapier. I'm assuming from this that you were looking to use Zapier as the "one place" (i.e in the middle)"
Now I don't know Zapier, but came across your question looking along the same lines. The terms I'm familiar with on this are "one-to-many", "fan-out" or "publish/subscribe" communication styles - but aware that these may be a bit old school in Web 2.0 world.
In my search I came across a Zapier Support post on Multiple Zaps.
In summary, looks like you may be able to achive this using a "Zapier Delay Trigger zap"
The possible downside (depending on delivery time requirements) is that the delivery of onward webhooks would be complete "...within 5-15 minutes". This doesn't quite fit my requirement, by may fit yours.
Edit: A quick search on "fan-out" term also found this on using Zapier Queue. Again,as it's queuing I suspect there may be a delay in onward processing.
Edit2: Following email to Zapier support - situation is still as per links i.e.time delay, but "...in your zap you can define the length of the delay to minimize the lag. This case you shouldn't have any problem setting a 1 minute delay since there's no need for it to be any longer, which can help you get a lot closer to 'near real time'."
Edit3: Further info from Zapier support - apparently if you configure two Webhook-Webhook Zaps you can fire them both with one URL as "https://zapier.com/hooks/catch/xxhook1,xyhook2" - appears to fire both zaps pretty immediately.
Edit4: One thing to watch with the multi-zap approach (as Edit3) is that if one send to target fails then you can replay manually (via Zapier site and paying customer) - there's no ability (currently) to manage automatic retrying (not sure if there's any ability to monitor this event).
I'm not after any code in particular but I want to know what is the most efficient way to build a function that will constantly check for updates for things such as messages e.g. Have a chat conversation window and I want live updates such as Facebook.
Currently I have implemented it by putting a while loop in my core code that checks if the view is currently visible run a Task every 5 seconds to get new messages. This works but I don't believe its the most efficient way to do it and I need to consider battery life. *Note I do change visibility when the view goes away e.g. on iOS i do
public override ViewDidDissapper {
Model.SetVisible(false)
}
Has anyone implemented some sort of polling on a cross platform app?
There are many different possible solutions here - which one you prefer depends a lot on your requirements in terms of latency, reliability, efficiency, etc - and it depends on how much you can change server side.
If your server is fixed as a normal http server, then frequent polling may be your best route forwards, although you could choose to modify the 5 seconds occasionally when you think updates aren't likely.
One step up from this is that you could try long polling http requests within your server.
Another step beyond that are using Socket (TCP, UDP or websocket) communications to provide "real time" messaging.
And in parallel to these things, you could also consider using PUSH notifications both within your app and in the background.
Overall, this is a big topic - I'd recommend reading up about PushSharp from #Redth and about SignalR from Microsoft - #gshackles has some blog posts about using this in Xamarin. Also, services like AzureMobileServices, UrbanAirship, Buddy, Parse, etc may help
I wanna read out a server status webpage every x seconds.
Site is: http://www.ffxiv-status.com/
how to do it easily and fast?
I found this lines in Google, to read out the page source:
WebClient wClient = new WebClient();
string strSource = wClient.DownloadString("http://www.google.de");
I could split the page then wth string and so on and can have a look of the status and ms of the server.
Is that the fastes and resource best way or is there a better way?
(c#.net 3.5 VS 2010)
What you're trying to make is a small scraper. You can definitely get the page source and then analyze it. I suggest using regular expressions to look for the desired content. Of course you'll have to keep updating your parsing routines if the page design changes.
HTTP has support for a request header, If-Modified-Since, which will only download a page if it has changed since some specified date in the past. Why don't you use that to only retrieve the full status page in the event it bears new information?
Also, if you are polling every five seconds, something is wrong. You should probably set up a push notification system to avoid a constant waste of system resources.
It's every 5 seconds. Who cares. It's not going to take 5s, that's for sure. I'm not even sure if the CPU will wake from sleep mode to handle this process.
It's processing will be dominated by network traffic, and I use the word "dominated" very, very lightly.
If you were running this on an HP calculator with Kermit, you might want to think about some other potential optimizations. But, for real systems...just not an issue.
If it were me, I'd be looking at "wget" and grep/sed to find and clean up the data. But I'm lazy.
i have always wanted to make a private server but i don't know how i would do this.
i know how a private server works, the game sends data packets to the server. the server will take the data and process it and send data to the other games connected.
my questions are,
how do you edit the game so it will go to your server/change game data.
how do you find what packets do what.
the game will be something like WOW, i have not desided yet.
If you are hoping to embark on creating your own MMORPG then you have a huge task ahead of you, and unfortunately to put it nicely you are probably being too ambitious especially if you are asking these sorts of questions.
You should probably read up on client server architecture.
Also, in answer to your questions about the structure of the data being sent and how it is interpreted, well, that's 100% up to the people that design the system. You will want to simulate the entire game on the server(s) and don't trust the clients at all.
For something as complex as a MMORPG it is really important to create a solid design for the system before anything else, this is very important.
Just to be clear your intent is to create an emulated MMO server to the effect of WOW?
That's not really a trivial task and carries with it its own ethical implications.
Just to get started will require a ton of research, inspection, decoding, an extreme attention to detail.
If you are serious about it, then I would suggest looking up networking tools that can help you inspect traffic across the network and creating a scientific process for operation inspection.
Again, it should be noted this is by no means a trivial task.
This will be fairly difficult as you do not have the communicaton protocol specification for the game's client/server communication.
If you want to start this, then create a server that is simply a pass through. That is, all client requests are forwarded to the particular server. Once you have generated a large enough sample size of packets to study, then you can begin to dissect the meaning of each byte (possibly). Of course, if the packets are encrypted in any way (even a simple XOR encryption) then you will have an even harder time trying to figure out what each byte means. You should capture a sample set using two clients running sniffers so you can see what happens when one client does something and it needs to be sent to all clients.
But if I were you, I would just abandon the idea and work on something else. My two cents..
If you'd like an inside look at how games do networking, there's always Ryzom, which went open-source earlier this year. If you're creating your own MMO you can begin right there, and if you're looking to reverse-engineer one you can practice with your own client and server.
We require all requests for downloads to have a valid login (non-http) and we generate transaction tickets for each download. If you were to go to one of the download links and attempt to "replay" the transaction, we use HTTP codes to forward you to get a new transaction ticket. This works fine for a majority of users. There's a small subset, however, that are using Download Accelerators that simply try to replay the transaction ticket several times.
So, in order to determine whether we want to or even can support download accelerators or not, we are trying to understand how they work.
How does having a second, third or even fourth concurrent connection to the web server delivering a static file speed the download process?
What does the accelerator program do?
You'll get a more comprehensive overview of Download Accelerators at wikipedia.
Acceleration is multi-faceted
First
A substantial benefit of managed/accelerated downloads is the tool in question remembers Start/Stop offsets transferred and uses "partial" and 'range' headers to request parts of the file instead of all of it.
This means if something dies mid transaction ( ie: TCP Time-out ) it just reconnects where it left off and you don't have to start from scratch.
Thus, if you have an intermittent connection, the aggregate transfer time is greatly lessened.
Second
Download accelerators like to break a single transfer into several smaller segments of equal size, using the same start-range-stop mechanics, and perform them in parallel, which greatly improves transfer time over slow networks.
There's this annoying thing called bandwidth-delay-product where the size of the TCP buffers at either end do some math thing in conjunction with ping time to get the actual experienced speed, and this in practice means large ping times will limit your speed regardless how many megabits/sec all the interim connections have.
However, this limitation appears to be "per connection", so multiple TCP connections to a single server can help mitigate the performance hit of the high latency ping time.
Hence, people who live near by are not so likely to need to do a segmented transfer, but people who live in far away locations are more likely to benefit from going crazy with their segmentation.
Thirdly
In some cases it is possible to find multiple servers that provide the same resource, sometimes a single DNS address round-robins to several IP addresses, or a server is part of a mirror network of some kind. And download managers/accelerators can detect this and apply the segmented transfer technique across multiple servers, allowing the downloader to get more collective bandwidth delivered to them.
Support
Supporting the first kind of acceleration is what I personally suggest as a "minimum" for support. Mostly, because it makes a users life easy, and it reduces the amount of aggregate data transfer you have to provide due to users not having to fetch the same content repeatedly.
And to facilitate this, its recommended you, compute how much they have transferred and don't expire the ticket till they look "finished" ( while binding traffic to the first IP that used the ticket ), or a given 'reasonable' time to download it has passed. ie: give them a window of grace before requiring they get a new ticket.
Supporting the second and third give you bonus points, and users generally desire it at least the second, mostly because international customers don't like being treated as second class customers simply because of the greater ping time, and it doesn't objectively consume more bandwidth in any sense that matters. The worst that happens is they might cause your total throughput to be undesirable for how your service operates.
It's reasonably straight forward to deliver the first kind of benefit without allowing the second simply by restricting the number of concurrent transfers from a single ticket.
I believe the idea is that many servers limit or evenly distribute bandwidth across connections. By having multiple connections, you're cheating that system and getting more than your "fair" share of bandwidth.
It's all about Little's Law. Specifically each stream to the web server is seeing a certain amount of TCP latency and so will only carry so much data. Tricks like increasing the TCP window size and implementing selective acks help but are poorly implemented and generally cause more problems than they solve.
Having multiple streams means that the latency seen by each stream is less important as the global throughput increases overall.
Another key advantage with a download accelerator even when using a single thread is that it's generally better than using the web browsers built in download tool. For example if the web browser decides to die the download tool will continue. And the download tool may support functionality like pausing/resuming that the built-in brower doesn't.
My understanding is that one method download accelerators use is by opening many parallel TCP connections - each TCP connection can only go so fast, and is often limited on the server side.
TCP is implemented such that if a timeout occurs, the timeout period is increased. This is very effective at preventing network overloads, at the cost of speed on individual TCP connections.
Download accelerators can get around this by opening dozens of TCP connections and dropping the ones that slow to below a certain threshold, then opening new ones to replace the slow connections.
While effective for a single user, I believe it is bad etiquette in general.
You're seeing the download accelerator trying to re-authenticate using the same transaction ticket - I'd recommend ignoring these requests.
From: http://askville.amazon.com/download-accelerator-protocol-work-advantages-benefits-application-area-scope-plz-suggest-URLs/AnswerViewer.do?requestId=9337813
Quote:
The most common way of accelerating downloads is to open up parllel downloads. Many servers limit the bandwith of one connection so opening more in parallel increases the rate. This works by specifying an offset a download should start which is supported for HTTP and FTP alike.
Of course this way of acceleration is quite "unsocial". The limitation of bandwith is implemented to be able to serve a higher number of clients so using this technique lowers the maximum number of peers that is able to download. That's the reason why many servers are limiting the number of parallel connection (recognized by IP), e.g. many FTP-servers do this so you run into problems if you download a file and try to continue browsing using your browser. Technically these are two parallel connections.
Another technique to increase the download-rate is a peer-to-peer-network where different sources e.g. limited by asynchron DSL on the upload-side are used for downloading.
Most download 'accelerators' really don't speed up anything at all. What they are good at doing is congesting network traffic, hammering your server, and breaking custom scripts like you've seen. Basically how it works is that instead of making one request and downloading the file from beginning to end, it makes say four requests...the first one downloads from 0-25%, the second from 25-50%, and so on, and it makes them all at the same time. The only particular case where this helps any, is if their ISP or firewall does some kind of traffic shaping such that an individual download speed is limited to less than their total download speed.
Personally, if it's causing you any trouble, I'd say just put a notice that download accelerators are not supported, and have the users download them normally, or only using a single thread.
They don't, generally.
To answer the substance of your question, the assumption is that the server is rate-limiting downloads on a per-connection basis, so simultaneously downloading multiple chunks will enable the user to make the most of the bandwidth available at their end.
Typically download-accelerators depend on partial content download - status code 206. Just like the streaming media players, media players ask for a small chunk of the full file to the server and then download it and play. Now the catch is if a server restricts partial-content-download then the download accelerator won't work!. It's easy to configure a server like Nginx to restrict partial-content-download.
How to know if a file can be downloaded via ranges/partially?
Ans: check for a header value Accept-Ranges:. If it does exist then you are good to go.
How to implement a feature like this in any programming language?
Ans: well, it's pretty easy. Just spin up some threads/co-routines(choose threads/co-routines over processes in I/O or network bound system) to download the N-number of chunks in parallel. Save the partial files in the right position in the file. and you are technically done. Calculate the download speed by keeping a global variable downloaded_till_now=0 and increment it as one thread completes downloading a chunk. don't forget about mutex as we are writing to a global resource from multiple thread so do a thread.acquire() and thread.release(). And also keep a unix-time counter. and do math like
speed_in_bytes_per_sec = downloaded_till_now/(current_unix_time-start_unix_time)