Html template that dynamically updates with new data from changed file - html

How to make html view/template that updates with new data from changed file
I am developing a chat messenger service and am storing messages in a SQL database i have made a similar file checking function like above question.
Is there anyway to add the output of this function into the html every time a new change to the database is detected?

You haven't gone in to much detail on what you're using to do this, but I'm going to answer and assume you're working with SocketIO.
Is there anyway to add the output of this function into the html every time a new change to the database is detected?
This is probaby the wrong way to think about this. The question you linked deals with tailing a file, and then acting appropriately. You shouldn't need to detect a database change, since your event function should be handling all the messages.
I created a gist of a very basic chat app which includes the functionality you desicribe. (Based on a tutorial which is linked in the README).
You should have a function which handles an incoming message from a user, then emits it on the socket, so other users on the system will see that message. This is where you want to update your database:
#socketio.on('my event')
def handle_my_custom_event(json, methods=['GET', 'POST']):
""" handle an incoming chat message, add to db, then emit to room """
# Unique ID for the session
json['sid'] = request.sid
print('received my event: ' + str(json)) # Just prints to server terminal
# Build the database entry and commit
new_db_entry = ChatHistory(sid = json['sid'],
message = json['message'],
username= json['user_name'])
db_session.add(new_db_entry)
db_session.commit()
# Escape nasty html. Do this for each var the user enters
json['user_name'] = su.escape(json['user_name'])
json['message'] = su.escape(json['message'])
socketio.emit('my response', json, callback=messageReceived)
You then have some corresponding Javascript which recieves this event in the frontend, and updates the HTML:
socket.on( 'my response', function( msg ) {
if( typeof msg.user_name !== 'undefined' ) {
$( 'div.message_holder' ).append( '<div><b style="color: #000">'+msg.user_name+'</b> '+msg.message+'</div>' )
}
})
However how do you render the history in a user's browser when they first connect? You could write some Javascript to do this, however my example simply queries the database when a user first renders the session.html template (the frontend):
#app.route('/')
def sessions():
history = ChatHistory.query.all()
return render_template('session.html', history=history)
And then in the corresponding template, builds the page at load time:
<div class="message_holder">
{% for chatItem in history %}
<div><b style="color: #000">{{ chatItem.username }}</b> {{ chatItem.message }}</div>
{% endfor %}
</div>
This is just a sample of what can be done.

Related

Site returns login page again when scraping after logging in successfully once using MechanicalSoup?

Im trying to scrape some data from Twitter using BeautifulSoup as a part of a project. To scrape the ‘following’ section I need to first login, so I tried doing so using MechanicalSoup. I know the login is successful as I received an email saying so, but when I go to a different page in the same website to scrape data it again redirects me to the login page.
import mechanicalsoup
browser = mechanicalsoup.StatefulBrowser(soup_config={'features': 'lxml'},
raise_on_404=True,
user_agent='MyBot/0.1: mysite.example.com/bot_info',)
login_page = browser.get("https://twitter.com/login")
login_form = login_page.soup.findAll("form")
login_form = login_form[2]
login_form.find("input", {"name": "session[username_or_email]"})["value"] = "puturusername"
login_form.find("input", {"name": "session[password]"})["value"] = "puturpassword"
login_response = browser.submit(login_form, login_page.url)
login_response.soup()
This sent me a successful login email, upon which I tried:
page_stml = browser.open('https://twitter.com/MKBHD/following').text
page_soup = soup(page_html,"html.parser")
page_soup
I received the page containing https://twitter.com/login?redirect_after_login=%2FMKBHD%2Ffollowing&amp instead of the actual ‘following’ page.
And if I try the code given below instead of 'browser.open('https://twitter.com/MKBHD/following').text':
# verify we are now logged in
page = browser.get_current_page()
print(page)
messages = page.find("div", class_="flash-messages")
if messages:
print(messages.text)
assert page.select(".logout-form")
print(page.title.text)
# verify we remain logged in (thanks to cookies) as we browse the rest of
# the site
page3 = browser.open("https://github.com/MechanicalSoup/MechanicalSoup")
assert page3.soup.select(".logout-form”)
I get the output:
----> 4 messages = page.find("div", class_="flash-messages")
AttributeError: 'NoneType' object has no attribute ‘find’
update:
the login_response.soup()
gives me the following:
</style>, <body>
<noscript>
<center>If you’re not redirected soon, please use this link.</center>
</noscript>
<script nonce="O1gf092z/sXmKkH64mLOzQ==">
document.cookie = "app_shell_visited=1;path=/;max-age=5";
location.replace(location.href.split("#")[0]);
</script>
</body>, <noscript>
<center>If you’re not redirected soon, please use this link.</center>
</noscript>, <center>If you’re not redirected soon, please use this link.</center>, use this link, <script nonce="O1gf092z/sXmKkH64mLOzQ==">
document.cookie = "app_shell_visited=1;path=/;max-age=5";
location.replace(location.href.split("#")[0]);
</script>]
To avoid to get the redirection page, you can use StatefulBrowser() object instead of Browser().
I wrote a short post about it : https://piratefache.ch/python-3-mechanize-and-beautifulsoup
import mechanicalsoup
if __name__ == "__main__":
URL = "https://twitter.com/login"
LOGIN = "your_login"
PASSWORD = "your_password"
TWITTER_NAME = "displayed_name" # Displayed username on Twitter
# Create a browser object
browser = mechanicalsoup.StatefulBrowser()
# request Twitter login page
browser.open(URL)
# we grab the login form
browser.select_form('form[action="https://twitter.com/sessions"]')
# print form inputs
browser.get_current_form().print_summary()
# specify username and password
browser["session[username_or_email]"] = LOGIN
browser["session[password]"] = PASSWORD
# submit form
response = browser.submit_selected()
# get current page output
response_after_login = browser.get_current_page()
# verify we are now logged in ( get img alt element containing username )
# if you found a better way to check, let me know. Since twitter generate dynamically all theirs classes, its
# pretty complicated to get better information
user_element = response_after_login.select_one("img[alt="+TWITTER_NAME+"]")
# if username is in the img field, it means the user is successfully connected
if TWITTER_NAME in str(user_element):
print("You're connected as " + TWITTER_NAME)
else:
print("Not connected")
Sources:
StatefulBrowser
Example with StatefulBrowser

Passing query strings from bot to html page

In Microsoft Bot Framework - botbuilder v.3.15.0
When opening url from bot without any query strings - it perfectly works - and opens external url (previously defined var urlExt, or just plain 'https://some.html') when clicking on a button in the bot... - in Bot Framework Emulator, Azure Web Chat, Messenger & Telegram - perfectly ok.
lib.dialog('/', [
function (session, args) {
args = args || {};
// var pkey1 = 'fdsa'; // dummy variable, url with querystring with this dummy works ok on all channels!
// var rkey1 = 'asdf'; // dummy variable, url with querystring with this dummy works ok on all channels!
var pkey1 = session.logger.address.conversation.id;
var rkey1 = session.logger.address.user.id;
console.log(pkey1); // correctly shows conversation id in console locally
console.log(rkey1); // correctly shows user id in console locally
var urlMap = `https://mymap.azurewebsites.net/?cid=${pkey1}&uid=${rkey1}`;
var urlExt = encodeURI(urlMap);
setTimeout(function () {
var addressCard = new builder.HeroCard(session)
.title('address_title')
.subtitle('address_subtitle')
.images([
new builder.CardImage(session)
.url('https://somedb.blob.core.windows.net/images/ab_small.png')
.alt('Here comes some pic')
])
.buttons([
builder.CardAction.openUrl(session, urlExt, 'Just do it!')
]);
session.send(new builder.Message(session)
.addAttachment(addressCard));
}, 1000)
},
function (session, results) {
// some further code
}
]);
But when you try to insert query string into urlExt - by taking it's parameters conversation.id and user.id from 'session' - by making variables who take values of conversation.id and user.id from 'session' and then inserting those variables into urlExt (either by '+' concatenation, or by `` ${} method) it works locally in Emulator - but not on Azure Web Chat or Messenger or Telegram.
When I try to find the reason for this problem I tried not to grab conversation.id or user.id from 'session', but just insert some dummy variables with text to forward them into html page by inserting those variables as part of the query string - IT WORKS...:(
Really strange, the problem seems to be with getting conversation.id & user.id from 'session' to variables at Azure Portal.
But why does it perfectly work locally on my laptop?
Logs on Azure say that:
TypeError: Cannot read property 'conversation' of undefined
I have looked in stackoverflow - there is ZERO info about it, I looked at the various code at GitHub - NO ANSWERS...
Please, hint, or help...
The session logger is only enabled on certain channels, which is why your code works in the emulator but not in Test in WebChat on Azure and Microsoft Teams. Instead of getting the conversation and user ids from the logger attribute, access them from the message property.
var pkey1 = session.message.address.conversation.id;
var rkey1 = session.message.address.user.id;
Hope this helps!

How to measure how many TCP connections were made in a page

With chrome dev tool, I can see the number of requests in a page, but it seems that there is no way to measure number of connections.
Is it possible in chrome dev tool? if not, what tools can I use instead?
You can enable the Connection ID header in the Network panel, which is a unique identifier for a particular connection. You can sort the column to see how many requests there were for a particular connection instance, but there's no built in way to see how many or filter the results.
However, this data can be exported into a JSON formatted file, known as the HAR (HTTP Archive). You can do this by right-clicking on panel and selecting 'Save as HAR with Content'.
You can extract the data from the JSON, and filter and aggregate however you like. I have created a simple example script that will load the HAR from the local file system, parse the data, and filter the content, so that it shows how many unique Connection IDs appeared in the session.
function loadFile(event) {
var file = event.target.files[0];
if (file) {
var reader = new FileReader();
reader.onload = function(e) {
var contents = e.target.result;
var data = JSON.parse(contents);
getUniqueConnectionCount(data);
}
reader.readAsText(file);
} else {
alert('Failed to load file.');
}
}
function getUniqueConnectionCount(data) {
var entries = data.log.entries;
var uniqueConnectionIds = entries.map(function(item) {
return item['connection'];
}).filter(function(x, i, a) {
return a.indexOf(x) === i && i > 0;
});
console.log('There were ', uniqueConnectionIds.length, ' unique connections found', uniqueConnectionIds);
}
document.getElementById('files').addEventListener('change', loadFile, false);
<div>
<input type='file' id='files' name='files' />
</div>
Note: Make sure 'Preserve Log' is un-checked to avoid seeing data from previous sessions. This is just a quick example for your use case, but I might look into extending this to be more generic.
It depends what connections you are interested in. You can use Chrome Developer tools the way Gideon Pyzer pointed out to see html connections. But if you are interested in TCP or some other protocol you can use Wireshark (free and open-source packet analyzer) to capture those connections.
Then there is "Capture Network Log" in chrome. Type "chrome://net-export/" in address field, set ready and press "Start Loggin to Disk" button - it will save your browser network activity to json file.

EWS - FileAttachment Content is empty / byte[0]

I have written a WebAPI controller method that finds a mail by its unique ID from ExchangeOnline. I wrote a small model class in order to store some attributes of a mail like the subject, the sender, the date received and so on.
Now I also want to access file attachments if the mail has such attachments. Therefore, I wrote this code (just the relevant part):
List<AttachmentItem> attDataContainer = new List<AttachmentItem>();
EmailMessage originalMail = EmailMessage.Bind(service, new ItemId(uniqueID), new PropertySet(ItemSchema.Attachments));
foreach (Attachment att in originalMail.Attachments)
{
if (att is FileAttachment && !att.IsInline)
{
FileAttachment fa = att as FileAttachment;
fa.Load();
attDataContainer.Add(
new AttachmentItem
{
ID = fa.Id,
Name = fa.Name,
ContentID = fa.ContentId,
ContentType = fa.ContentType,
ContentLocation = fa.ContentLocation,
Content = Convert.ToBase64String(fa.Content),
Size = fa.Size
});
}
}
The method indeed finds the attachments and displays all of the attributes you can see in the "AttachmentItem" object - BUT NOT the fa.Content attribute.
I have crwaled almost any document I could find on this (especially the *.Load() part as well as much examples. But in my case I get "byte[0]" when debugging the output.
Do you have any idea for me what could be the reason for this?
PS: By the way, I have version v2.0.50727 of Microsoft.Exchange.WebServices referenced.
Best regards and thanks in advance,
Marco
When you call the load method on the Attachment that should make a GetAttachment request to the server which will return the data for that Attachment. If you enable tracing https://msdn.microsoft.com/en-us/library/office/dn495632(v=exchg.150).aspx that should allow you to follow the underlying SOAP requests which should help with troubleshooting (eg you can see what the server is returning which is important).
The other thing to check is that is this is a real attachment vs a One Drive attachment which could be the case on ExchangeOnline.

SSIS - In DFT, how to each row input to exe and read the output for each row?

In the Data Flow Task, I need re-direct each row to exe and get the output for each row, similar to script component.
I tried to use Process in Script Component, but it is throwing exception "StandardOut has not been redirected or the process hasn't started yet.".
The code used in Script Component is:
Process myApp = new Process();
myApp.StartInfo.FileName = #"Someexe.exe";
myApp.StartInfo.Arguments = "param1";
myApp.StartInfo.UseShellExecute = false;
myApp.StartInfo.RedirectStandardOutput = false;
myApp.Start();
while (!myApp.HasExited)
{
string result= myApp.StandardOutput.ReadToEnd();
}
Any suggestions? I would like to know if there is any better way to do this?
Your error message is helpful: "StandardOut has not been redirected..."
Since you want to capture the output, redirecting it to the Destination Component, don't you want to change the line:
myApp.StartInfo.RedirectStandardOutput = false;
to be:
myApp.StartInfo.RedirectStandardOutput = true;
Consider the example at this link, on BOL:
ProcessStartInfo.RedirectStandardOutput Property