How can a website call a file that no longer exists? - html

I have a website. I want to update my CSS file. After 20 minutes, uploading it 20 times, every browser still pulls the same CSS file. I delete the file off the server. There is literally no CSS file. I clear all my browser's cache's and history; everything. I load the website. It still pulls the old CSS file. Observations:
This is obviously a browser issue: when I load from mobile, the new file is there. So how can I give my Browser's a hint and actually make them clear cookies/cache etc?
Now I go to another computer that has never accessed the website, and it pulls old CSS file.
Please explain to me how, and why this is possible. Is it my host? I am using Netfirms, but how long can it possibly take to update a CSS file?

Its perfectly normal for browsers to cache content unless you take complete control over how long you want your pages and their resources to get cached.
What you want to do in these cases is implement something called "cache busting". What cache busting does is force browsers to treat a changed file on the server as a new request by simply making the browsers think its a different file or request.
The easiest way to do that is to append a query string like ?v=123 to your resource (JavaScript, CSS) URL. But do take care that some proxies will ignore to refresh their cached content if you use query strings. Therefore my preferred way of handling cache busting is to use rewrite rules to point requests for files like style-1391836063.css to a file which always has the same name on the server, i.e. style.css. The rewrite rule for that can be as simple as
RewriteRule (.+)-(\d+).(css|js)$ $1.$3 [L]
The whole magic is in the 1391836063 part which is actually a timestamp generated with filemtime(). That timestamp represents the last time the file was changed so it will retain the same value as long as style.css stays the same. In that case browsers will simply see the same resource name and will have no need to redownload its since its already cached and considered up to date.
However, if style.css does change the modiefied timestamp will also change which will result in a different numerical part, i.e. style-1391867247.css. If that happens browsers will be forced to discard any cached data and treat that request as a completely new resource.
However, for this to work you would also need server side support or in other words a script which will be doing all the fingerprinting (fingerprinting is actually the correct way to reference this technique). No matter what server side technology you use the process would be as follows.
With a DOM parser you look up for all the references to CSS and/or JavaScript files.
For every reference you find you check if the file exists on the server and if it does you read its modified timestamp with filemtime(). Then you append it to the actual resource name (style.css becomes something like style-1391867247.css)
You returne the fingerpinted code to the browser.
Here is a PHP class I wrote to perform fingeprinting on most of my projects. Please note that in order to avoid unnecessary processing this class should be used with some form of server-side caching to avoid fingeprinting if its not required. Additionally the code references some external constants and classes which should be disregarded as the code simply tries to demonstrate one way of how fingerprinting can be done.
/**
* CacheBusting class responsible for fingerprinting CSS and JavaScript resources in order to prevent caching issues.
*/
class CacheBuster {
/**
* Parses the loaded (X)HTML code and fingerprints all resources with their Last Modified timestamp.
*
* #param string $content XHTML content to fingerprint.
* #return mixed Either fingerprinted $content on success or false on failure.
*/
public static function fingerprint($content){
/**
* ExtendedDOMDocument for manipulating content data (something written by me to replace the gimpy DOMDocument class)
*
* #var ExtendedDOMDocument
*/
$dom;
/**
* XPath responsible for handling $dom.
*
* #var DOMXPath
*/
$xpath;
/**
* List of extracted DOM nodes.
*
* #var DOMNodeList
*/
$nodes;
/**
* Helper variable containing current resource URI.
*
* #var string
*/
$resource = '';
/**
* Helper variable containing all the results from regex matches.
*
* #var array
*/
$matches = array();
/**
* Array of resource URIs with their corresponding fingerprint versions.
*
* #var array
*/
$fingerprints = array();
// In case $content is not provided false is returned.
if(!strlen($content)){
return false;
}
// Loading $content into DOMDocument parser.
$dom = new ExtendedDOMDocument();
$dom->loadHTML($content);
// Extracting <script> and <link> nodes.
$xpath = new DOMXPath($dom);
$nodes = $xpath->query('//script|//link');
// Traversing the extracted nodes in order to find out the exact names of the CSS and JavaScript resources and then create the appropriate fingerprint.
foreach($nodes as $node){
//Only local resources with specified "src" or "href"" are taken into account.
switch($node->getAttribute('type')){
case 'text/javascript' :
$resource = $node->getAttribute('src');
break;
case 'text/css' :
$resource = $node->getAttribute('href');
break;
default:
// In case no type is specified we probe for either "src" or "href" but if nothing is found we skip this node.
/**
* Value of the 'src' attribute for the current node, if any.
*
* #var string
*/
$src = $node->getAttribute('src');
/**
* Value of the 'href' attribute for the current node, if any.
*
* #var string
*/
$href = $node->getAttribute('href');
if(strlen($src) && strpos($src, '.js')){
$resource = $src;
} else if(strlen($href) && strpos($href, '.css')) {
$resource = $href;
} else {
//No luck? Skipping the current node.
continue;
}
}
// Generating fingerprint pairs.
if(!strlen(trim($resource)) || (stripos($resource, 'http://') !== false) || (stripos($resource, 'https://') !== false)){
// Skipping inline and remote scripts.
continue;
} else {
// Fingerprinting resources...
preg_match('/(.+)\.(css|js)/', $resource, $matches);
if(file_exists(APP_FOLDER . $matches[0])){ // Resource exists.
$fingerprints[] = array(
'original' => $resource,
'fingerprinted' => $matches[1] . '-' . filemtime(APP_FOLDER . $matches[0]) . '.' . $matches[2]
);
}
}
}
// Time to make fingerprint pair replacements.
foreach($fingerprints as $pair){
$content = str_replace($pair['original'], $pair['fingerprinted'], $content);
}
return $content;
}
}
You can use the class as simple as calling it anywhere in the code.
$output = CacheBuster::fingerprint($MARKUP);

I think this is a classic case off "unintentionall" serverside caching. That meens that the server keeps a copy of your files to shorten the clients loading time.
Depening on the caching solution you could possibly turn the caching of with .htaccess.
Hope this helps!

Related

QWebEngineView - loading of > 2mb content

So, using PyQt5's QWebEngineView and the .setHTML and .setContent methods have a 2 MB size limitation. When googling for solutions around this, I found two methods:
Use SimpleHTTPServer to serve the file. This however gets nuked by a firewall employed in the company.
Use File Urls and point to local files. This however is a rather bad solution, as the HTML contains confidential data and I can't leave it on the harddrive, under any circumstance.
The best solution I currently see is to use file urls, and get rid of the file on program exit/when loadCompleted reports it is done, whichever comes first.
This is however not a great solution and I wanted to ask if there is a solution I'm overlooking that would be better?
Why don't you load/link most of the content through a custom url scheme handler?
webEngineView->page()->profile()->installUrlSchemeHandler("app", new UrlSchemeHandler(e));
class UrlSchemeHandler : public QWebEngineUrlSchemeHandler
{ Q_OBJECT
public:
void requestStarted(QWebEngineUrlRequestJob *request) {
QUrl url = request->requestUrl();
QString filePath = url.path().mid(1);
// get the data for this url
QByteArray data = ..
//
if (!data.isEmpty())
{
QMimeDatabase db;
QString contentType = db.mimeTypeForFileNameAndData(filePath,data).name();
QBuffer *buffer = new QBuffer();
buffer->open(QIODevice::WriteOnly);
buffer->write(data);
buffer->close();
connect(request, SIGNAL(destroyed()), buffer, SLOT(deleteLater()));
request->reply(contentType.toUtf8(), buffer);
} else {
request->fail(QWebEngineUrlRequestJob::UrlNotFound);
}
}
};
you can then load a website by webEngineView->load(new QUrl("app://start.html"));
All relative pathes from inside will also be forwarded to your UrlSchemeHandler..
And rember to add the respective includes
#include <QWebEngineUrlRequestJob>
#include <QWebEngineUrlSchemeHandler>
#include <QBuffer>
One way you can go around this is to use requests and QWebEnginePage's method runJavaScript:
web_engine = QWebEngineView()
web_page = web_engine.page()
web_page.setHtml('')
url = 'https://youtube.com'
page_content = requests.get(url).text
# document.write writes a string of text to a document stream
# https://developer.mozilla.org/en-US/docs/Web/API/Document/write
# And backtick symbol(``) is for multiline strings
web_page.runJavaScript('document.write(`{}`);'.format(page_content))

Google Translate free api assist

I am kind of new to javascript and building websites, I program c# most of the times.
I am trying to build something and I need to use google translate api, the problem that is cost money so I prefer use Free API so I found this.
https://ctrlq.org/code/19909-google-translate-api
so I changed it a bit and tried alone, because I wasn't sure what e type ment.
so this is my code:
function doGet(text) {
var sourceText = text;
var translatedText = LanguageApp.translate('en', 'iw', sourceText);
var urllog = "https://translate.googleapis.com/translate_a/single?client=gtx&sl="
+ "en" + "&tl=" + "iw" + "&dt=t&q=" + encodeURI(text);
var result = JSON.parse(UrlFetchApp.fetch(urllog).getContentText());
translatedText = result[0][0][0];
console.log(translatedText);
}
so the url is downloading me a text file called "f.txt" that include the translate code the problem is that I doesnt want it to download File,
I just need the translate inside the txt file its gives me,
also the problem is I am not sure how to get that info inside a javascript variable, And I doesnt want it to give me that file as well..
So how Can I read it?
how can I use the file without download it, and How can I push it to a string variable?
And How I can cancel the download and get only the translate?
THANKS!
By the way
and if anyone know the function doGet(e) that I showed on the link, what is "e"? what does the function wants?
I know I'm a year late but I came to same problem and fixed it using PHP. I have created this simple PHP function:
function translate($text, $from, $to) {
if($text == null)
echo "Please enter a text to translate.";
if($from == null)
$from = "auto";
if($to == null)
$to = "en";
$NEW_TEXT = preg_replace('/\s+/', '+', $text);
$API_URL = "https://translate.googleapis.com/translate_a/single?client=gtx&sl=" . $from . "&tl=" . $to . "&dt=t&q=" . $NEW_TEXT;
$OUTPUT = get_remote_data($API_URL);
$json = json_decode($OUTPUT, true); // decode the JSON into an associative array
$TRANSLATED_OUTPUT = $json[0][0][0];
echo $TRANSLATED_OUTPUT;
}
Example usage (English to Spanish):
translate("Hello", "en", "es"); //Output: Hola
/*
sourceLanguage: the 2-3 letter language code of the source language (English = "en")
targetLanguage: the 2-3 letter language code of the target language (Hebrew is "iw")
text: the text to translate
callback: the function to call once the request finishes*
* Javascript is much different from C# in that it is an asynchronous language, which
means it works on a system of events, where anything may happen at any time
(which makes sense when dealing with things on the web like sending requests to a
server). Because of this, Javascript allows you to pass entire
functions as parameters to other functions (called callbacks) that trigger when some
time-based event triggers. In this case, as seen below,
we use our callback function when the request to google translate finishes.
*/
const translate = function(sourceLanguage,targetLanguage,text,callback) {
// make a new HTTP request
const request = new XMLHttpRequest();
/*
when the request finishes, call the specified callback function with the
response data
*/
request.onload = function() {
// using JSON.parse to turn response text into a JSON object
callback(JSON.parse(request.responseText));
}
/*
set up HTTP GET request to translate.googleapis.com with the specified language
and translation text parameters
*/
request.open(
"GET",
"https://translate.googleapis.com/translate_a/single?client=gtx&sl=" +
sourceLanguage + "&tl=" + targetLanguage + "&dt=t&q=" + text,
true
);
// send the request
request.send();
}
/*
translate "This shouldn't download anything" from English to Hebrew
(when the request finishes, it will follow request.onload (specified above) and
call the anonymous
function we use below with the request response text)
*/
translate("en","iw","This shouldn't download anything!",function(translation) {
// output google's JSON object with the translation to the console
console.log(translation);
});

Parsing large JSON file using symfony controller

I have a large json file (not in size, but in elements). It has 30000 JSON elements and I am trying to produce Entities from it as it reads it.
So far I have it reading the file with Guzzle, and it ends up producing about 1500 entities before it crashes. I feel that I must be doing this the wrong way.
Here is my code:
public function generateEntities(Request $request, $number)
{
$client = new \GuzzleHttp\Client();
$request = new \GuzzleHttp\Psr7\Request('GET', 'http://www.example.com/file.json');
$promise = $client->sendAsync($request)->then(function ($response) {
$batchSize = 20;
$i = 0;
foreach (json_decode($response->getBody()) as $entityItem) {
$entity = new Entity();
$entity->setEntityItem($entityItem->string);
$em = $this->getDoctrine()->getManager();
$em->persist($entity);
if (($i % $batchSize) === 0) {
$em->flush(); // Executes all updates
}
$i++;
}
$em->flush();
});
$promise->wait();
return $this->redirect($this->generateUrl('show_entities'));
}
I worked out from research that I should be clearing the Entity Manager frequently, so I added in batch sizing etc to flush it every 20 entities created. This did help but is not enough to load the full 30000 files.
Maybe I am completely wrong and should be handling it a different way?
Is it possible for someone to please point me in the right direction, I am happy to work it out on my own I am just a little unsure where to proceed from here.
Thank you!
You could improve your process act in two ways:
1) increment the time limit of the execution of the controller action with the function set_time_limit, so put this as first line of the controller:
public function generateEntities(Request $request, $number)
{
set_time_limit(0); // set to zero, no time limit is imposed
2) Free much memory is possible for each interaction flush the data to the database and detach/free memory as follow:
$em->persist($entity);
$em->flush($entity);
$em->detach($entity);
$em->clear($entity);
unset($entity);
Hope this help
The script is running out of memory because every one of the 30000 entities is managed in memory. You need to detach the entities from the manager periodically to make sure they are "garbage collected". Use $em->clear(); in your batch flushing block to ensure memory isn't exhausted. See the Doctrine page on batch operations for more information.
Keep in mind though, that $em->clear() will detach all entities from the manager, not just those you are using in this loop.

How to publish an asset within a widget in Yii2

In Yii 1 it was possible to publish an asset with:
Yii::app()->getAssetManager()->publish(Yii::getPathOfAlias('ext.MyWidget.assets'));
How can I publish an asset within a widget in Yii2?
In your view of your widget:
app\assets\AppAsset::register($this); // $this == the View object
Check the docs.
Simplest way of doing this:
app\assets\AppAsset::register($this->getView());
It is the same way you would publish it in a view.
The problem for this case in the question.
In Yii2 AssetBandle should be unique because it avoids duplication on load same files on a web page. In Yii1 it was a problem.
Almost all of the suggested answers here do not solve this problem.
That why the answer should be like this:
You should create new asset bundle(yii\web\AssetBundle) with a unique name that globally identifies it among all asset bundles used in an application. The unique name will be checked by fully qualified class name. After uses follow:
YourAsset::register($view);
or
$view->registerAssetBundle(InterrogateDeviceButtonAsset::class);
Nearly the same :
Yii::$app->getAssetManager()->publish('/path/to/assets'); // you can use an alias
http://www.yiiframework.com/doc-2.0/yii-web-assetmanager.html#publish()-detail
But if you want to publish css and js, it should be better to create an asset bundle, read more here : http://www.yiiframework.com/doc-2.0/guide-structure-assets.html#defining-asset-bundles
I have implemented this function in my BaseController.php which is extended by all my controllers. It's a function to publish a single asset.
/**
* Publish an asset and return url
*
* #param $src
*
* #return mixed
*/
public function publishAsset( $src ) {
$path = Yii::getAlias( $src );
if ( ! $this->assetManager ) {
$this->assetManager = new AssetManager();
}
$return = $this->assetManager->publish( $path );
return $return[1];
}
The only thing I'm trying to adjust is making it possible to include the full URL scheme. I've posted my question about that here Yii2 AssetManager published path include URL scheme

Magento Front End 404 Error

I am a newbie to magento, I have installed and put a few products, but then later I was getting Error:404 page not found in the front end. Backend is all OK, I am able to access everything but all of a sudden I don't know how this happened. I tried all the solutions like Flush Cache, replacing .htaccess, is_active field in database etc but all proved futile. Then lately I have put in system->Configuration->Web Base_url as http://sportiva.no/index.php/ (Previously it was http://sportiva.no/) and all is completely changed all the styles went away and I am not able to save anything. Please help, I am ready to give backend credentails.
Please help
Go to System > Configuration > Web > Default Pages and check "CMS Home Page" field value. If it is "404 Not Found", then change it to any of the CMS page available on the drop-down and save the configuration.
Refere to this link by Alan Storm,
http://magento-quickies.alanstorm.com/post/6462298634/404-debugging
excerpt from it,
Put this code in function _validateControllerClassName
<?php
#File: app/code/core/Mage/Core/Controller/Varien/Router/Standard.php
/**
* Generating and validating class file name,
* class and if evrything ok do include if needed and return of class name
*
* #return mixed
*/
protected function _validateControllerClassName($realModule, $controller)
{
$controllerFileName = $this->getControllerFileName($realModule, $controller);
if (!$this->validateControllerFileName($controllerFileName)) {
var_dump($controllerFileName);
return false;
}
$controllerClassName = $this->getControllerClassName($realModule, $controller);
if (!$controllerClassName) {
var_dump($controllerClassName);
return false;
}
// include controller file if needed
if (!$this->_includeControllerClass($controllerFileName, $controllerClassName)) {
var_dump($controllerFileName . '----' . $controllerClassName);
return false;
}
return $controllerClassName;
}