I'm using Jinja2 with a custom loader that creates (complex) templates dynamically. This means that error messages do not contain a template filename but instead refer to the "top-level template code". To inspect errors, I would like to print the top level template code to the console (for example by calling render in a try-catch block). How can this be achieved?
One (not so elegant) way to achieve this is to cache all responses of the custom loader:
import os
_template_by_fn = dict()
def print_template(template_fn):
print("Template: %s" % template_fn)
print(os.linesep.join(_template_by_fn[str(template_fn)]))
def custom_load_template(template_fn):
# Do work to create the template.
template_code = ""
# When done, cache the result.
_template_by_fn[str(template_fn)] = template_code
return template_code
I have my website with thwo hrefs and under this hrefs i have my flask function download_file(). It's working but now I add on my function in flask a parameter. So now I want to run something like that download_file(1) when someone clicked on first href and download_file(2) when someone clicked on href2
<h2>Download a file </h2>
<p>
Download1
Download2
</p>
############ FLASK CODE
from flask import Flask, send_file, render_template
app = Flask(__name__)
#app.route('/')
def download():
return render_template('indexdownload.html')
#app.route('/download')
def download_file(number):
p = "test"
if number == 1:
p = "1.txt"
elif number == 2:
p = "2.txt"
return send_file(p, as_attachment=True)
app.run()
I try different ways but I always get some syntax errors because I don't know hot to do that.
An anchor with an href attribute sends a GET request.
There are several alternative ways to send data within a GET request.
Variable Rule
One way to pass data within a URL is to use variable rules. The values are embedded within the path, extracted server-side, and passed to the endpoint as arguments.
#app.route('/download/<int:num>')
def download(num):
# ...
URL Parameter
An alternative is to pass it as a URL parameter. In this case, the data must be requested within the endpoint and are not necessarily available.
The respective parameter is queried using the name of the object request.args. It is possible to define a default value and optionally to automatically convert the value by specifying a type.
#app.route('/download')
def download():
num = request.args.get('num', 0, type=int)
# ...
In both cases you can pass the value to the jinja2 statement url_for to form the url.
url_for('download', num=1)
Based on the code within the request and the intended use, I would suggest the first variant. So you are assuming that a value is specified. You can put your filename together and deliver the file.
Incidentally, you can also pass an entire file name within the URL.
#app.route('/download/<path:filename>')
def download(filename):
# ...
Download
I'm working on some universal solution for problem with static files and updates in it.
Example: let's say there was site with /static/styles.css file - and site was used for a long time - so a lot of visitors cached this file in browser
Now we doing changes in this css file, and update on server, but some users still have old version (despite modification date returned by server)
The obvious solution is to add some version to file /static/styles.css?v=1.1 but in this case developer must track changes in this file and manually increase version
A second solution is to count the md5 hash of the file and add it to the url /static/styels.css/?v={mdp5hashvalue} which looks much better, but md5 should be calculated automatically somehow.
they possible way I see it - create some template tag like this
{% static_file "style.css" %}
which will render
<link src="/static/style.css?v=md5hash">
BUT, I do not want this tag to calculate md5 on every page load, and I do not want to store hash in django-cache, because then we will have to clear after updating file...
any thoughts ?
Django 1.4 now includes CachedStaticFilesStorage which does exactly what you need (well... almost).
Since Django 2.2 ManifestStaticFilesStorage should be used instead of CachedStaticFilesStorage.
You use it with the manage.py collectstatic task. All static files are collected from your applications, as usual, but this storage manager also creates a copy of each file with the MD5 hash appended to the name. So for example, say you have a css/styles.css file, it will also create something like css/styles.55e7cbb9ba48.css.
Of course, as you mentioned, the problem is that you don't want your views and templates calculating the MD5 hash all the time to find out the appropriate URLs to generate. The solution is caching. Ok, you asked for a solution without caching, I'm sorry, that's why I said almost. But there's no reason to reject caching, really. CachedStaticFilesStorage uses a specific cache named staticfiles. By default, it will use your existing cache system, and voilà! But if you don't want it to use your regular cache, perhaps because it's a distributed memcache and you want to avoid the overhead of network queries just to get static file names, then you can setup a specific RAM cache just for staticfiles. It's easier than it sounds: check out this excellent blog post. Here's what it would look like:
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.PyLibMCCache',
'LOCATION': '127.0.0.1:11211',
},
'staticfiles': {
'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
'LOCATION': 'staticfiles-filehashes'
}
}
I would suggest using something like django-compressor. In addition to automatically handling this type of stuff for you, it will also automatically combine and minify your files for fast page load.
Even if you don't end up using it in entirety, you can inspect their code for guidance in setting up something similar. It's been better vetted than anything you'll ever get from a simple StackOverflow answer.
I use my own templatetag which add file modification date to url: https://bitbucket.org/ad3w/django-sstatic
Is reinventing the wheel and creating own implementation that bad? Furthermore I would like low level code (nginx for example) to serve my staticfiles in production instead of python application, even with backend. And one more thing: I'd like links stay the same after recalculation, so browser fetches only new files. So here's mine point of view:
template.html:
{% load md5url %}
<script src="{% md5url "example.js" %}"/>
out html:
static/example.js?v=5e52bfd3
settings.py:
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(PROJECT_DIR, 'static')
appname/templatetags/md5url.py:
import hashlib
import threading
from os import path
from django import template
from django.conf import settings
register = template.Library()
class UrlCache(object):
_md5_sum = {}
_lock = threading.Lock()
#classmethod
def get_md5(cls, file):
try:
return cls._md5_sum[file]
except KeyError:
with cls._lock:
try:
md5 = cls.calc_md5(path.join(settings.STATIC_ROOT, file))[:8]
value = '%s%s?v=%s' % (settings.STATIC_URL, file, md5)
except IsADirectoryError:
value = settings.STATIC_URL + file
cls._md5_sum[file] = value
return value
#classmethod
def calc_md5(cls, file_path):
with open(file_path, 'rb') as fh:
m = hashlib.md5()
while True:
data = fh.read(8192)
if not data:
break
m.update(data)
return m.hexdigest()
#register.simple_tag
def md5url(model_object):
return UrlCache.get_md5(model_object)
Note, to apply changes an uwsgi application (to be specific a process) should be restarted.
Django 1.7 added ManifestStaticFilesStorage, a better alternative to CachedStaticFilesStorage that doesn't use the cache system and solves the problem of the hash being computed at runtime.
Here is an excerpt from the documentation:
CachedStaticFilesStorage isn’t recommended – in almost all cases ManifestStaticFilesStorage is a better choice. There are several performance penalties when using CachedStaticFilesStorage since a cache miss requires hashing files at runtime. Remote file storage require several round-trips to hash a file on a cache miss, as several file accesses are required to ensure that the file hash is correct in the case of nested file paths.
To use it, simply add the following line to settings.py:
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.ManifestStaticFilesStorage'
And then, run python manage.py collectstatic; it will append the MD5 to the name of each static file.
How about you always have a URL Parameter in your URL with a version and whenever you have a major release you change the version in your URL Parameter. Even in the DNS. So if www.yourwebsite.com loads up www.yourwebsite.com/index.html?version=1.0 then after the major release the browser should load www.yourwebsite.com/index.html?version=2.0
I guess this is similar to your solution 1. Instead of tracking files can you track whole directories? For example ratehr than /static/style/css?v=2.0 can you do /static-2/style/css or to make it even granular /static/style/cssv2/.
There is an update for #deathangel908 code. Now it works well with S3 storage also (and with any other storage I think). The difference is using of static file storage for getting file content. Original doesn't work on S3.
appname/templatetags/md5url.py:
import hashlib
import threading
from django import template
from django.conf import settings
from django.contrib.staticfiles.storage import staticfiles_storage
register = template.Library()
class UrlCache(object):
_md5_sum = {}
_lock = threading.Lock()
#classmethod
def get_md5(cls, file):
try:
return cls._md5_sum[file]
except KeyError:
with cls._lock:
try:
md5 = cls.calc_md5(file)[:8]
value = '%s%s?v=%s' % (settings.STATIC_URL, file, md5)
except OSError:
value = settings.STATIC_URL + file
cls._md5_sum[file] = value
return value
#classmethod
def calc_md5(cls, file_path):
with staticfiles_storage.open(file_path, 'rb') as fh:
m = hashlib.md5()
while True:
data = fh.read(8192)
if not data:
break
m.update(data)
return m.hexdigest()
#register.simple_tag
def md5url(model_object):
return UrlCache.get_md5(model_object)
The major advantage of this solution: you dont have to modify anything in the templates.
This will add the build version into the STATIC_URL, and then the webserver will remove it with a Rewrite rule.
settings.py
# build version, it's increased with each build
VERSION_STAMP = __versionstr__.replace(".", "")
# rewrite static url to contain the number
STATIC_URL = '%sversion%s/' % (STATIC_URL, VERSION_STAMP)
So the final url would be for example this:
/static/version010/style.css
And then Nginx has a rule to rewrite it back to /static/style.css
location /static {
alias /var/www/website/static/;
rewrite ^(.*)/version([\.0-9]+)/(.*)$ $1/$3;
}
Simple templatetag vstatic that creates versioned static files urls that extends Django's behaviour:
from django.conf import settings
from django.contrib.staticfiles.templatetags.staticfiles import static
#register.simple_tag
def vstatic(path):
url = static(path)
static_version = getattr(settings, 'STATIC_VERSION', '')
if static_version:
url += '?v=' + static_version
return url
If you want to automatically set STATIC_VERSION to the current git commit hash, you can use the following snippet (Python3 code adjust if necessary):
import subprocess
def get_current_commit_hash():
try:
return subprocess.check_output(['git', 'rev-parse', '--short', 'HEAD']).strip().decode('utf-8')
except:
return ''
At settings.py call get_current_commit_hash(), so this will be calculated only once:
STATIC_VERSION = get_current_commit_hash()
I use a global base context in all my views, where I set the static version to be the millisecond time (that way, it will be a new version every time I restart my application):
# global base context
base_context = {
"title": settings.SITE_TITLE,
"static_version": int(round(time.time() * 1000)),
}
# function to merge context with base context
def context(items: Dict) -> Dict:
return {**base_context, **items}
# view
def view(request):
cxt = context({<...>})
return render(request, "page.html", cxt)
my page.html extends my base.html template, where I use it like this:
<link rel="stylesheet" type="text/css" href="{% static 'style.css' %}?v={{ static_version }}">
fairly simple and does the job
For web2py there are generic views e.g. for JSON.
I could not find a sample.
When looking at the web2py manual 10.1.2 and 10.1.6, its written:
'.. define a "generic.csv" file, but one would have to specify the name of the object to be serialized ("animals" in the example)'
Looking at the generic pdf view
{{
import os
from gluon.contrib.generics import pdf_from_html
filename = '%s/%s.html' % (request.controller,request.function)
if os.path.exists(os.path.join(request.folder,'views',filename)):
html=response.render(filename)
else:
html=BODY(BEAUTIFY(response._vars))
pass
=pdf_from_html(html)
}}
and also the specified csv (Manual charpter 10.1.6):
{{
import cStringIO
stream=cStringIO.StringIO() animals.export_to_csv_file(stream)
response.headers['Content-Type']='application/vnd.ms-excel'
response.write(stream.getvalue(), escape=False)
}}
Massimo is writing: 'web2py does not provide a "generic.csv";'
He is not fully against it but..
So lets try to get it and deactivate when necessary.
The generic view should look similar to (the non working)
(well, this we better call pseudocode as it is not working):
{{
import os
from gluon.contrib.generics export export_to_csv_file(stream)
filename = '%s/%s' % (request.controller,request.function)
if os.path.exists(os.path.join(request.folder,'views',filename)):
csv=response.render(filename)
else:
csv=BODY(BEAUTIFY(response._vars))
pass
= export_to_csv_file(stream)
}}
Whats wrong?
Or is there a sample?
Is there a reson not to have a generic csv?
{{
import os
from gluon.contrib.generics export export_to_csv_file(stream)
filename = '%s/%s' % (request.controller,request.function)
if os.path.exists(os.path.join(request.folder,'views',filename)):
csv=response.render(filename)
else:
csv=BODY(BEAUTIFY(response._vars))
pass
= export_to_csv_file(stream)
}}
Adapting the generic.pdf code so literally as above would not work for CSV output, as the generic.pdf code is first executing the standard HTML template and then simply converting the generated HTML to a PDF. This approach does not make sense for CSV, as CSV requires data of a particular structure.
As stated in the documentation:
Notice that one could also define a "generic.csv" file, but one would
have to specify the name of the object to be serialized ("animals" in
the example). This is why we do not provide a "generic.csv" file.
The execution of a view is triggered by a controller action returning a dictionary. The keys of the dictionary become available as variables in the view execution environment (the entire dictionary is also available as response._vars). If you want to create a generic.csv view, you therefore need to establish some conventions about what variables are in the returned dictionary as well as the possible structure(s) of the returned data.
For example, the controller could return something like dict(data=mydata). The code in generic.csv would then access the data variable and could convert it to CSV. In that case, there would have to be some convention about the structure of data -- perhaps it could be required to be a list of dictionaries or a DAL Rows object (or optionally either one).
Another possible convention is for the controller to return something like dict(columns=mycolumns, rows=myrows), where columns is a list of column names and rows is a list of lists containing the data for each row.
The point is, there is no universal convention for what the controller might return and how that can be converted into CSV, so you first need to decide on some conventions and then write generic.csv accordingly.
For example, here is a very simple generic.csv that would work only if the controller returns dict(rows=myrows), where myrows is a DAL Rows object:
{{
import cStringIO
stream=cStringIO.StringIO() rows.export_to_csv_file(stream)
response.headers['Content-Type']='application/vnd.ms-excel'
response.write(stream.getvalue(), escape=False)
}}
I tried:
# Sample from Web2Py manual 10.1.1 Page 464
def count():
session.counter = (session.counter or 0) + 1
return dict(counter=session.counter, now = request.now)
#and my own creation from a SQL table (if possible used for json and csv):
def csv_rt_bat_c_x():
battdat = db().select(db.csv_rt_bat_c.rec_time, db.csv_rt_bat_c.cellnr,
db.csv_rt_bat_c.volt_act, db.csv_rt_bat_c.id).as_list()
return dict(battdat=battdat)
Bot times I get an error when trying csv. It works for /default/count.json but not for /default/count.csv
I suppose the requirement:
dict(rows=myrows)
"where myrows is a DAL Rows object" is not met.
I am new to use Jinja2 and try to insert the current date in a document as a bottom line to tell users when the document was produced.
My current solution is
Produced on {{ utils.today|date('%x') }}
There are no error messages, but nothing is produced.
The solution need to be Jinja2 only, as I have no python process running - using Ginger (a Haskell program) to process the template.
Context Processors can be used to inject values into templates before rendering it.
In app.py:
import datetime
#app.context_processor
def inject_today_date():
return {'today_date': datetime.date.today()}
And add this in the html file:
<p>{{today_date}}</p>
Output: 2019-01-07
Jinja2 doesn't have native support for inserting the date. However, you can easily prepare the date in whatever library you use to render the template and pass the value to the rendering function.
Using jinja2 in Python:
import datetime
date = datetime.date.today()
template = Template('Produced on {{ date }}')
template.render(date=date)
If you are using Ginger, you would have the same template, just make sure to create the date in Haskell and render the template with that value.
You can also write or install a jinja2 extension that will make utilities for working with dates available in the template.
For example, after installing jinja2-time [1], your template would look like this:
Produced on {% now 'local' %}
[1]: untested code