Django dynamic table with AJAX - html

This is the second question in the row about my Django project. The code I use here has parts copied from this question:Stack Overflow
What I aim to achieve is a dynamic table, that loops through objects in a list(currently the interval between records is 3 seconds). Let's say I have 21 records. The table first displays records of 1 to 10, on the displaykala.html table. Then it replaces only the table content with records of 11 to 20(with AJAX, without page refresh. The new contents come from get_more_tables_kala.html, and those table rows coming from there, are appended to the displaykala.html table).
At no point should the table be empty (UNLESS there is simply no objects to display).
Finally, the contents of 11-20 are replaced with contents of 21-30. In this case, only one row is displayed.
Basicly it always displays first ten records(even if there is only one). Then the code increments the startindex and endindex of the rows, and checks if there is records between those. If there is more than 10 records, it will empty the table and instantly load up the next records (as long as there is even 1 record). Else, the program should wait X amount of seconds, until checking again.
The code has been going through various experimentations, I'm sorry for any dumb coding. I'm still learning Django and AJAX.
There is a few problems though, but let's focus on the main one
When the displaykala.html is loaded, the table is displayed empty first, for 3 seconds. I know this is because the code loads the empty displaykala.html table.
If I were to set in views that displaykala.html table should display rows 1 to 10 by default, I run into a rendering problem if there is more than 10 rows.
Those rows are loaded from the get_more_tables_kala.html, but when the code is supposed to switch back to displaykala.html, I'm either forced to reload the page (which is not an option due to network traffic increase), or return a new render with displaykala.html as a parameter, which causes the page to create a "copy" of itself, in the place where the table rows are supposed to be.
What I want is the program to switch between the records, without diplaying empty page in between.
I am open to any kind of optimizations or different ideas, as long as they're somewhat simple to understand. I know that this code is rubbish. I'm just trying to get it to work.
I must have missed something vital to tell you, please comment below if you need more info.
EDIT: Also, when I look at the django server console, the three tables (1-10, 11-20(only two records) and the empty one) produce these rows:
[14/Dec/2017 12:33:22] "GET /user/get_more_tables_k HTTP/1.1" 200 2222
[14/Dec/2017 12:33:24] "GET /user/get_more_tables_k HTTP/1.1" 200 439
[14/Dec/2017 12:33:27] "GET /user/get_more_tables_k HTTP/1.1" 200 1
My code runs between views.py, js_kala.html, displaykala.html and get_more_tables_kala.html.
views.py
from django.shortcuts import render, redirect
from userside.models import Kala
from django.contrib.auth.decorators import login_required
from django.db import connection
#login_required
def displaykala(request):
return render(request, 'displaykala.html')
#login_required
def get_more_tables_kala(request):
startind = request.session.get('startind')
if not startind:
startind = 0
request.session['startind'] = startind
endind = request.session.get('endind')
if not endind:
endind = 10
request.session['endind'] = endind
kalat = Kala.objects.filter(rivinro__gt=startind, rivinro__lte=endind)
count = kalat.count()
if count == 0:
request.session['startind'] = 0
request.session['endind'] = 10
kalat = Kala.objects.filter(rivinro__gt=startind, rivinro__lte=endind)
return render(request, 'get_more_tables_kala.html', {'kalat': kalat})
else:
request.session['startind'] += 10
request.session['endind'] += 10
kalat = Kala.objects.filter(rivinro__gt=startind, rivinro__lte=endind)
return render(request, 'get_more_tables_kala.html', {'kalat': kalat})
js_kala.html
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"> </script>
<script>
var append_increment = 0;
setInterval(function() {
$.ajax({
type: "GET",
url: "{% url 'get_more_tables_kala' %}", // URL to your view that serves new info
data: {'append_increment': append_increment}
})
.done(function(response) {
$('#_appendHere_kala').html('');
$('#_appendHere_kala').append(response);
append_increment += 10;
});
}, 3000)
</script>
displaykala.html
<html>
<META HTTP-EQUIV="PRAGMA" CONTENT="NO-CACHE">
{% load static %}
{% include 'loginbar.html' %}
<head>
<link rel="stylesheet" type="text/css" href="{% get_static_prefix %}css/style.css">
<title>Display</title>
</head>
<body>
<h1>Display</h1>
<table>
<tr>
<th>Rivinumero</th>
<th>Tuote</th>
<th>Latinankielinen nimi</th>
<th>Pyyntialue</th>
<th>Pyyntipäivä</th>
<th>Tuotantotapa</th>
<th>Pyydystyyppi</th>
</tr>
</table>
<table id="_appendHere_kala" class="table table-striped table-condensed">
<thead>
</thead>
{% include 'js_kala.html' %}
{% for kala in kalat %}
<tr>
<tbody id="tbody">
<td>{{kala.rivinro}}</td>
<td>{{kala.tuote}}</td>
<td>{{kala.latinalainen_nimi}}</td>
<td>{{kala.pyyntialue}}</td>
<td>{{kala.pyyntipaiva|date:"d.m.Y" }}</td>
<td>{{kala.tuotantotapa}}</td>
<td>{{kala.pyydystyyppi}}</td>
</tbody>
</tr>
{% endfor %}
</table>
</body>
</html>
get_more_tables_kala.html
{% load static %}
{% for kala in kalat %}
<tbody id='tbody'>
<tr>
<td>{{kala.rivinro}}</td>
<td>{{kala.tuote}}</td>
<td>{{kala.latinalainen_nimi}}</td>
<td>{{kala.pyyntialue}}</td>
<td>{{kala.pyyntipaiva|date:"d.m.Y" }}</td>
<td>{{kala.tuotantotapa}}</td>
<td>{{kala.pyydystyyppi}}</td>
</tr>
</tbody>
{% endfor %}

You should fetch data from database in "displaykala" view and throw the data into the template file. The startind should be 0 and endind should be 9 because you are fetching 10 records only.
For the empty table problem, it seems that the setInterval() function would cause the table to be empty for 3 seconds. After calling the setInterval(), it will wait for a period of time (3 seconds in this case) before executing the process. Which means this function makes your program cannot be executed immediately.
Here is the process workflow:
Wait for 3 seconds
Run AJAX
Wait for 3 seconds
Run AJAX
Again and again
Which means you should execute the AJAX process once before calling the setInterval().
--Edit--
After asking so many questions, I've found something that might also cause the empty table problem.
In this part,
if count == 0:
request.session['startind'] = 0
request.session['endind'] = 10
kalat = Kala.objects.filter(rivinro__gt=startind, rivinro__lte=endind)
return render(request, 'get_more_tables_kala.html', {'kalat': kalat})
You still use startind and endind (which are the same arguments used in previous filtering statement) to do filter when the system cannot find any data in the filtering. This makes your system prints nothing.
However, from your code, I cannot see why your system finds nothing from the database in every time you started opening the page. You will have to open debug mode of IDE and DevTools of web browser to see what happens with the session and Python variables.

Solved for now, got help from elsewhere with a brand new solution.

Related

selenium by_xpath not returning any results

I am using Selenium 4+, and I seem to not get back the any result when requesting for elements in a div.
# Wait for the page to load
wait = WebDriverWait(driver, 10)
wait.until(EC.presence_of_element_located((By.ID, "search-key")))
# wait for the page to load
driver.implicitly_wait(10)
# search for the suppliers you want to message
search_input = driver.find_element(By.ID,"search-key")
search_input.send_keys("suppliers")
search_input.send_keys(Keys.RETURN)
# find all the supplier stores on the page
supplier_stores_div = driver.find_element(By.CLASS_NAME, "list--gallery--34TropR")
print(supplier_stores_div)
supplier_stores = supplier_stores_div.find_elements(By.XPATH, "./a[#class='v3--container--31q8BOL cards--gallery--2o6yJVt']")
print(supplier_stores)
The logging statements gave me <selenium.webdriver.remote.webelement.WebElement (session="a3be4d8c5620e760177247d5b8158823", element="5ae78693-4bae-4826-baa6-bd940fa9a41b")> and an empty list for the Element objects, []
The html code is here:
<div class="list--gallery--34TropR" data-spm="main" data-spm-max-idx="24">flex
<a class="v3--container--31q8BOL cards--gallery--2o6yJVt" href="(link)" target="_blank" style="text-align: left;" data-spm-anchor-id="a2g0o.productlist.main.1"> (some divs) </a>flex
That is just one <a> class, there's more.
Before scraping the supplier names, you have to scroll down the page slowly to the bottom, then only you can get all the supplier names, try the below code:
driver.get("https://www.aliexpress.com/premium/supplier.html?spm=a2g0o.best.1000002.0&initiative_id=SB_20221218233848&dida=y")
last_height = driver.execute_script("return document.body.scrollHeight")
while True:
driver.execute_script("window.scrollBy(0, 800);")
sleep(1)
new_height = driver.execute_script("return document.body.scrollHeight")
if new_height == last_height:
break
last_height = new_height
suppliers = driver.find_elements(By.XPATH, ".//*[#class='list--gallery--34TropR']//span/a")
print("Total no. of suppliers:", len(suppliers))
print("======")
for supplier in suppliers:
print(supplier.text)
Output:
Total no. of suppliers: 60
======
Reading Life Store
ASONSTEEL Store
Paper, ink, pen and inkstone Store
Custom Stationery Store
ZHOUYANG Official Store
WOWSOCOOL Store
IFPD Official Store
The 9 Store
QuanRun Store
...
...
It returns you the Element object.
If you want to get the text you need to write
supplier_stores_div.find_elements(By.XPATH, "./a[#class='v3--container--31q8BOL cards--gallery--2o6yJVt']").getText()
or for getting an attribute (for example the href)
supplier_stores_div.find_elements(By.XPATH, "./a[#class='v3--container--31q8BOL cards--gallery--2o6yJVt']").getAttribute("href")

FLASK : call several routes for several functions in one HTML page

In the script below, I try to call via FLASK several functions in my homepage in HTML format. Each function calculates a number of pending that I want to display in a table on my homepage. To do this, I have:
computed with Panda a number of events to calculate from a csv file. I tested these functions separately and they work
created a table on my HTML homepage with the different categories for which I would like to display the number of events from my csv file
used the html_page.replace function to replace the number $$numberxx$$ in the html table by the result of my functions in Python
for each function, I created a route indicating the HTML link and the concerned function so that Flask can understand that for this function, after having calculated the result, it is necessary to go on the homepage page and replace in the table the object $$number1$$ by the result1 or $$number2$$ by the result2
Unfortunately, when I run Flask, nothing is happening. I still have $$number1$$ or $$number2$$ on my webpage.
Could you please help me to correct my script below:
Python code:
import flask
import csv
import pandas as pd
import numpy as np
app = flask.Flask("app_monitoringissues")
def get_html(page_name):
html_file = open(page_name + ".html")
content = html_file.read()
html_file.close()
return content
#app.route("/homepage/<count_pending1>")
def count_pending1():
html_page = get_html ("homepage")
df = pd.read_csv("sortdata.csv")
count1 = len(df[df["Status_Issue"].astype(str).str.contains("Pending-to be checked")])
count2 = df["Status_Issue"].isna().sum()
result1 = count1 + count2
return html_page.replace("$$NUMBER1$$", str(result1))
#app.route("/homepage/<count_pending2>")
def count_pending2():
html_page = get_html ("homepage")
df = pd.read_csv("sortdata.csv")
count1 = len(df[df["Status_Issue"].astype(str).str.contains("Pending CP")])
result2 = count1
return html_page.replace("$$NUMBER2$$", result2)
HTML code:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Homepage</title>
</head>
<body>
<div class = "head">
<h1 id = "title homepage"> monitoring screen</h1>
<div class = "tablesupervisionarea">
<table class="tablesupervision">
<thead class ="theadsupervision">
<tr>
<th>Pending Supervision</th>
<th> </th>
</tr>
</thead>
<tbody>
<tr>
<th>Need to be checked by Tax Regulatory</th>
<td><ol>$$NUMBER1$$</ol></td>
</tr>
<tr>
<th>Pending with CP</th>
<td><ol>$$NUMBER2$$</ol></td>
</tr>
<tr>
</tbody>
</table>
</div>
This is not how you use Flask. Use render_template instead. Have a look at the doc first to understand the concept.
As an example, this function:
#app.route("/homepage/<count_pending2>")
def count_pending2():
html_page = get_html ("homepage")
df = pd.read_csv("sortdata.csv")
count1 = len(df[df["Status_Issue"].astype(str).str.contains("Pending CP")])
result2 = count1
return html_page.replace("$$NUMBER2$$", result2)
should look like:
#app.route("/homepage/<count_pending2>")
def count_pending2():
df = pd.read_csv("sortdata.csv")
result = len(df[df["Status_Issue"].astype(str).str.contains("Pending CP")])
return render_template("homepage.html", result=result)
And in the relevant HTML template, add a tag like this that will be replaced with appropriate values:
{{ result }}
Have a look at the doc, it's easy you'll see.
If I may suggest, try to improve the naming of variables and function names: count_pending1/2 etc are all very similar and do not give any clue about the purpose of the function. The code should be more explicit - before you even look at the function it should be obvious what it is supposed to do.
When you need to review your code, it would help a lot to have meaningful names to immediately spot the relevant section you want to edit. You already have two functions that are named almost the same and are quite similar. Ask yourself if you really need two functions or even more. Perhaps a simple function with a conditional block would make more sense than repeating code and making the whole program longer than it could be.

Django not recognizing user form email verification link:

I've spent the last week on this like 20 hours and can't figure it out. This is my very first Django application.
I have tested the 'urlsafe_base64_encode(force_bytes(user.pk))' in shell and it works to pass the user's 'uid', however it does not work in the application itself. I get that the 'activation link is invalid' for all links even though the link passed the correct primary key, I'm not sure what is going wrong.
Please help.
views.py/registration page
from_email, subject = 'domain <info#domain>', 'Email verification'
htmly = get_template('activation_email_template.html')
username = user.username
message = render_to_string('activation_email.html', {
'user':user,
'token':account_activation_token.make_token(user),
'uid':urlsafe_base64_encode(force_bytes(user.pk)),
})
d = {'username': username, 'url':reverse('activate',
kwargs={
'token':account_activation_token.make_token(user),
'uidb64':urlsafe_base64_encode(force_bytes(user.uid))
})}
html_content = htmly.render(d)
msg = EmailMultiAlternatives(subject, message, from_email,[user.email])
msg.attach_alternative(html_content, "text/html")
msg.send()
here is my activate/views.py:
try:
uid = force_text(urlsafe_base64_decode(uidb64))
user = User.objects.get(pk=uid)
except(TypeError, ValueError, OverflowError, User.DoesNotExist):
user = None
if user is not None and account_activation_token.check_token(user, token):
...
else:
return HttpResponse('Activation link is invalid!')
For some reason I'm getting 'Activation Link is invalid' every time I try this. I know it's sending the right PK, because I can clink on the url and the url is correct with the correctly encoded pk. However, the activate view does not recognize this pk.
here is my activation_email.html:
{% autoescape off %}
Hi {{ user.username }},
Thanks for creating an account. Please click on the link to confirm your registration,
http://domain{% url 'activate' uidb64=uid token=token %}
{% endautoescape %}
This is the activation_email_template:
<h3>Hi <strong>{{ username }}</strong>,</h3>
<p>Thanks for creating an account. Please click on the following link to confirm your registration:</p>
<a style="background-color: #f79407 ;padding: 10px 15px; text-decoration: none;color: #ffffff;text-transform: uppercase;font-size: 15px;" href="https://domain{{ url }}">Email Activation</a>
Please help!
Tried this to get at errors in activation link, but still just get activation link is invalid:
try:
uid = force_text(urlsafe_base64_decode(uidb64))
user = User.objects.get(pk=uid)
except TypeError:
print('typerror')
user=None
except ValueError:
print('valueerror')
user=None
except OverflowError:
print('OverflowError')
user=None
except User.DoesNotExist:
print('user.DoesNotExist')
user=None
if user is not None and account_activation_token.check_token(user, token):
....
else:
raise ValidationError('user is none')

Typoscript: render HTML table with database values

Im completely new to typoscript therefore I have quite a hard time with the syntax but I think I am getting there.
My task is to render an HTML table and fill it with values from a database table (doesn't matter which one). In my case I took the tt_content table and tried to fill my HTML table with the "header" field and the "bodytext" field.
So I made a completely empty template and wrote the following code in the "setup" field of the template. I added some headers and texts to the sites I have to test my code but I get a completely empty page not even the "table" HTML tags are there.
After 4 days of research I still don't know what my problem is here so I am quite desperate.
Here is what I have so far:
page = PAGE
page.typeNum = 0
lib.object = COA_INT
lib.object {
10 = TEXT
10.value = <table>
20 = CONTENT
20.wrap = <tr>|</tr>
20 {
table = tt_content
select {
orderBy = sorting
}
renderObj = COA
renderObj {
10 = COA
10 {
10 = TEXT
10 {
field = header
wrap = <td>|</td>
}
20 = TEXT
20 {
field = bodytext
wrap = <td>|</td>
}
}
}
}
20 = TEXT
20.value = </table>
}
If someone could help me out here it would be much appreciated.
Thanks in advance.
Check if you have any 'template parser' running.
go to template -> choose 'Info/modify' and click on 'edit the whole ...'
There choose the includes tab and include css_styled_content' (Yes, there is another way of parsing your content, with fluid_styled_content'. you can choose that instead if you are on TYPO3 7.6.* or higher)
These 'parsers' will give you all the needed typoscript included to parse and render your content. Without these, nothing will be rendered when you want to render content from the backend.
second: your typoscript is wrong
You have made a content array (lib.content is a Content Object Array) and filled it with content. But you overwrite the content with key 20.
change
20 = TEXT
20.value = </table>
to
30 = TEXT
30.value = </table>
third: you have created a Page object but you did not add your COA into that page object.
Try this:
page = PAGE
page.10 < lib.object
What this does is include your lib.content in the Page Object at 'level' 10
you can also do
page.20 = TEXT
page.20.value = hello world
This will be rendered after your lib.content.
As you could notice. It is a bit as writing a big Array (because typoscript is a big Array ;)
Beware that you place your lib.content ABOVE the page object declaration. else it will not be able to include it.
There is also a slack channel for TYPO3 you can join if you have other questions. People over there are more then willing to help you.
https://forger.typo3.org/slack

page number variable: html,django

i want to do paging. but i only want to know the current page number, so i will call the webservice function and send this parameter and recieve the curresponding data. so i only want to know how can i be aware of current page number? i'm writing my project in django and i create the page with xsl. if o know the page number i think i can write this in urls.py:
url(r'^ask/(\d+)/$',
'ask',
name='ask'),
and call the function in views.py like:
ask(request, pageNo)
but i don't know where to put pageNo var in html page. (so fore example with pageN0=2, i can do pageNo+1 or pageNo-1 to make the url like 127.0.0.01/ask/3/ or 127.0.0.01/ask/2/). to make my question more cleare i want to know how can i do this while we don't have any variables in html?
sorry for my crazy question, i'm new in creating website and also in django. :">
i'm creating my html page with xslt. so i send the total html page. (to show.html which contains only {{str}} )
def ask(request:
service = GetConfigLocator().getGetConfigHttpSoap11Endpoint()
myRequest = GetConfigMethodRequest()
myXml = service.GetConfigMethod(myRequest)
myXmlstr = myXml._return
styledoc = libxml2.parseFile("ask.xsl")
style = libxslt.parseStylesheetDoc(styledoc)
doc = libxml2.parseDoc(myXmlstr)
result = style.applyStylesheet(doc, None)
out = style.saveResultToString( result )
ok = mark_safe(out)
style.freeStylesheet()
doc.freeDoc()
result.freeDoc()
return render_to_response("show.html", {
'str': ok,
}, context_instance=RequestContext(request))
i'm not working with db and i just receive xml file to parse it. so i don't have contact_list = Contacts.objects.all(). can i still use this way? should i put the first parameter inpaginator = Paginator(contact_list, 25) blank?
if you user standart django paginator, thay send you to url http://example.com/?page=N, where N - number you page
So,
# urls.py
url('^ask/$', 'ask', name='viewName'),
You can get page number in views:
# views.py
def ask(request):
page = request.GET.get('page', 1)