I am new to pyppeteer and try to download a csv. Unfortunately, without success. Suppose I want to download the following csv: 'https://people.sc.fsu.edu/~jburkardt/data/csv/addresses.csv'. Simply using
await self.page.goto('https://people.sc.fsu.edu/~jburkardt/data/csv/addresses.csv')
does not help. I tried to use
cdp = await self.page.target.createCDPSession()
await cdp.send('Page.setDownloadBehavior', { 'behavior': 'allow', 'downloadPath': '/test'})
but it's also not working. If someone knows how to make it work or can give me any advice, I'd highly appreciate that.
You seem to miss the _client module of the page object. I am not that expert in python, but it seems that will do the job without createCDPSession(), also the path is suspicious that it is not recognized, try an absolute one as downloadPath.
await self.page._client.send('Page.setDownloadBehavior', { 'behavior': 'allow', 'downloadPath': 'C:/test' })
await self.page.goto('https://people.sc.fsu.edu/~jburkardt/data/csv/addresses.csv')
Related
I have a nuxt-app, where I have a lot of posts. I recently refactored the project and I won't generate all posts anymore, as it is just taking too much time to do that.
Instead I have a page where I would fetch the matching post content via url query:
www.mypage.com/posts/?post=my-post-slug
Because the content is lying in static json files, for example in:
/static/data/posts/my-post-slug.json
/static/data/posts/my-post-slug_2.json
/static/data/posts/my-post-slug_3.json
/static/data/posts/my-post-slug_n.json
I read the post https://github.com/nuxt/nuxt.js/issues/123
about how to load json in the best way.
I decided to do something like this in the fetch() hook:
// ... simplified code
async fetch() {
let postSlug = this.$route.query.post
const post = this.$axios
.get(`/posts/posts.de.${postSlug}.json`)
.then((data) => {
return data?.data
})
.catch((error) => {
console.error('error: ', error)
const code = parseInt(error.response && error.response.status)
if (code === 404) {
this.$nuxt.error({ statusCode: 404, message: 'Post not found' })
}
})
this.activePost = post?.items?.[0] || false
}
As already said, I do not generate the actual posts, but I generate all post urls in my sitemap.xml.
When running the generate in analyze mode I have now a huuuuge bundle size (app.js), and I can't understand why...
-> Check out the attached image. (Note: app.js has a ridiculous size of 34MB!!!!😕)
I don't understand why all my post jsons appear in the static and the dist part of the bundle???
I don't understand at all why they appear in there. I want them to just lie in the static folder, but not be included in the app bundle.
(you can see that there are files like events.bundle.de.json included. I need those to fetch a list of all posts for example. I do that also within my fetch hook only.
I would be very happy if somebody could point out why those files are included (twice)!
Those files are not included "twice". You need them, so you do have them locally in your static folder.
Meanwhile, you should probably put them inside of your src directory if you don't want/need to have them exposed publicly and benefit from code-splitting thanks to Webpack as explained in the post you've linked (which is still valid even if it's a 2017 one!).
Here, since you're making an axios call and using target: 'static', it will bundle your whole thing to work even without JS and it does that ahead of time. So, in order to have all the possibilities, it includes all in the final bundle I think.
If you want to only load what is needed while not shipping a big static directory, you should import them dynamically. You can use a dynamic import: load only the needed JSON by passing the actual postSlug.
PS: style-wise, prefer using async/await (.then is deprecated) and maybe also $axios.$get.
Although I think #kissu s answer is answering my question in the title, it was not the solution for my problem. For the sake of completeness I will post what I found out after long and many hours of debugging. I still don't quite understand why this even happened, but maybe someone could comment on that as well:
In my nuxt-project I am using a utility file getData.js of which I import a function getDataServer into one of my vuex store modules.
// vuex store module: store/data.js
import { getPreviewData } from '~/api/getData'
The code looks like this:
// getData.js
// Get static JSON file (e.g. basic.de.json or posts.de.1.json)
export function getDataServer(fileProps) {
return require(`~/${fileProps.path}${fileProps.name}.${fileProps.lang}${
fileProps.page ? `.${fileProps.page}` : ''
}.json`)
}
Only by importing and not even by executing that function webpack would bundle EVERY .json file it can find in my root folder into my app.js.
That is why I had a dist folder appearing in my bundle, if not deleted. (The point I talk about in my original question, where I have things included twice).
I even created additional folders and .json files to see, and they were all bundled no matter what.
Only after removing the getData.js from my project my bundle became clean.
I understand that with the require command, webpack cannot tree-shake things, so I would have expected that some code-splitting features would not work, but what I did not expect was that this bit of code would automatically get every .json in my folder...
Does anyone know why importing that function would execute it in a way that acts as a wildcard for all .jsons?
To me it still does not make any sense.
Thanks and cheers.
I'm upgrading from an old version of PDF.JS (1.9.426) to the latest stable (2.5.207).
Previously, it was very simple to force the locale to match your applications current locale.
You simply pass it through as a parameter in the URL;
/PDFJS/web/viewer.html?file=example.pdf #locale=zh-CN
However, on upgrading, I see that this is no longer recommended.
This is despite the documentation still mentioning it.
In my application, users can override their prefered locale, so it might not match the browser.
I've looked into embed / object, which seemed like the solution, until I realised Android browsers don't support embedded PDF viewing.
I've been searching for the correct way to implement this, and so far I've not found a good or recommended answer.
It seems PDF.js is still the best all round solution, but I can't figure out the correct way of using it to simply render an existing PDF into a div.
From what I can tell the developers simply removed the option, didn't upgrade the docs, and are recommending to hard code the locale in viewer.js (which to me, kind of defeats the point).
I'm hoping someone can point me in the correct direction, and possibly help others who come across this.
For anyone looking to just get this working (until a better answer comes along), heres a fix;
Find the method "_initializeL10n" in viewer.js (line 552 in v2.5.207);
async _initializeL10n() {
this.l10n = this.externalServices.createL10n({
locale: _app_options.AppOptions.get("locale")
});
const dir = await this.l10n.getDirection();
document.getElementsByTagName("html")[0].dir = dir;
}
Change it to this;
async _initializeL10n() {
var hash = document.location.hash.substring(1);
var hashParams = (0, _ui_utils.parseQueryString)(hash);
this.l10n = this.externalServices.createL10n({
locale: ('locale' in hashParams) ? hashParams['locale'] : _app_options.AppOptions.get("locale")
});
const dir = await this.l10n.getDirection();
document.getElementsByTagName("html")[0].dir = dir;
}
This takes the code we need from the old version and gets #locale= working.
I hope to be told the correct way of doing it at some point.
I'm new to cypress and have ran into an issue. I have my base URL set to the domain I want to test, the issue is when I want to test the ability to login on my base url site I need to verify the user on another site, once I click apply on site number 2 the page on my base url reloads and I would then be able to test the rest of the site.
When I try to visit site 2 from my test I get an error
cy.visit() failed because you are attempting to visit a URL that is of
a different origin.
The new URL is considered a different origin because the following
parts of the URL are different:
superdomain
You may only cy.visit() same-origin URLs within a single test.
I read this https://docs.cypress.io/guides/guides/web-security.html#Set-chromeWebSecurity-to-false I've tried setting "chromeWebSecurity": false in cypress.json but I still get the same issue (I'm running in chrome)
Is there something I am missing?
As a temporary but solid work around, I was able to find this script in one of the Cypress Git issue threads (I don't remember where I found it so I can't link back to it)
Add the below to your cypress commands file
Cypress.Commands.add('forceVisit', url => {
cy.window().then(win => {
return win.open(url, '_self');
});
});
and in your tests you can call
cy.forceVisit("www.google.com")
From version 9.6.0 of cypress, you can use cy.origin.
If you want to use it, you must first set the "experimentalSessionAndOrigin" record to true.
{
"experimentalSessionAndOrigin": true
}
And here's how to use it.
cy.origin('www.example.com', () => {
cy.visit('/')
})
cy.origin change the baseurl, so you can link to another external link via cy.visit('/').
You can stub the redirect from login site to base site, and assert the URL that was called.
Based on Cypress tips and tricks here is a custom command to do the stubbing.
The login page may be using one of several methods to redirect, so besides the replace(<new-url>) stub given in the tip I've added href = <new-url> and assign(<new-url>).
Stubbing command
Cypress.Commands.add('stubRedirect', () => {
cy.once('window:before:load', (win) => {
win.__location = { // set up the stub
replace: cy.stub().as('replace'),
assign: cy.stub().as('assign'),
href: null,
}
cy.stub(win.__location, 'href').set(cy.stub().as('href'))
})
cy.intercept('GET', '*.html', (req) => { // catch the page as it loads
req.continue(res => {
res.body = res.body
.replaceAll('window.location.replace', 'window.__location.replace')
.replaceAll('window.location.assign', 'window.__location.assign')
.replaceAll('window.location.href', 'window.__location.href')
})
}).as('index')
})
Test
it('checks that login page redirects to baseUrl', () => {
cy.stubRedirect()
cy.visit(<url-for-verifying-user>)
cy.wait('#index') // waiting for the window load
cy.('button').contains('Apply').click() // trigger the redirect
const alias = '#replace' // or '#assign' or '#href'
// depending on the method used to redirect
// if you don't know which, try each one
cy.get(alias)
.should('have.been.calledOnceWith', <base-url-expected-in-redirect>)
})
You can't!
But, maybe it will be possible soon. See Cypress ticket #944.
Meanwhile you can refer to my lighthearted comment in the same thread where I describe how I cope with the issue while Cypress devs are working on multi-domain support:
For everyone following this, I feel your pain! #944 (comment) really gives hope, so while we're patiently waiting, here's a workaround that I'm using to write multi-domain e2e cypress tests today. Yes, it is horrible, but I hope you will forgive me my sins. Here are the four easy steps:
Given that you can only have one cy.visit() per it, write multiple its.
Yes, your tests now depend on each other. Add cypress-fail-fast to make sure you don't even attempt to run other tests if something failed (your whole describe is a single test now, and it makes sense in this sick alternate reality).
It is very likely that you will need to pass data between your its. Remember, we're already on this crazy “wrong” path, so nothing can stop us naughty people. Just use cy.writeFile() to save your state (whatever you might need), and use cy.readFile() to restore it at the beginning of your next it.
Sue me.
All I care about at this point is that my system has tests. If cypress adds proper support for multiple domains, fantastic! I'll refactor my tests then. Until that happens, I'd have to live with horrible non-retriable tests. Better than not having proper e2e tests, right? Right?
You could set the window.location.href manually which triggers a page load, this works for me:
const url = 'http://localhost:8000';
cy.visit(url);
// second "visit"
cy.window().then(win => win.location.href = url);
You will also need to add "chromeWebSecurity": false to your cypress.json configuration.
Note: setting the window to navigate won't tell cypress to wait for the page load, you need to wait for the page to load yourself, or use timeout on get.
I have a new bucket and 1 object inside. Object was uploaded from Revit file. When i change something on revit file and try to upload it again with the same name I always have the same old model on Forge as the first times, not thing was updated.
I use Nodejs and configure for the upload look like this.
let opts = {'xAdsForce': true}
try {
await new DerivativesApi().translate(job, opts, req.oauth_client, req.oauth_token);
} catch(err) {
next(err);
}
Is there something wrong ?. Thank in advance !
Passing in the xAdsForce: true option is the right way to force the translation even if the object with the same name has already been translated. According to the docs this method should return a promise that resolves in a Job object. Try investigating this value to see if everything is ok. Also make sure that the upload of your updated Revit file actually succeeded.
Alternatively, you can try triggering the translation manually, by making a request to the POST job endpoint, with x-ads-force header in the request set to true.
I have a single page web application using React and materialize-css and I would like to export it as static HTML and CSS so that it is possible to easily edit HTML for the purpose of prototyping. Is it possible to export at least a snapshot of current state?
I tried "save page" in Firefox and Chrome, but it does not provide good results.
Follow the following steps :-
1. In brouser, got to the developer tools,
2. select Inspector(firefox)/Elements(chrome),
3. then select the tag HTML, right click on it,
4. then click Edit as HTML.
Now you can copy all the code and save it. While the color and shape of the document remains, you will miss the pictures.
Good luck ! :)
Probably not ideal, but you can store the entire page as a variable and download it. Run this in your browser console after the page has loaded:
var pageHTML = document.documentElement.outerHTML;
var tempEl = document.createElement('a');
tempEl.href = 'data:attachment/text,' + encodeURI(pageHTML);
tempEl.target = '_blank';
tempEl.download = 'thispage.html';
tempEl.click();
The ReactDOMServer module contains a function for rendering a React application to static HTML - it's designed for use on the server, but I don't think there's anything to stop you using it in the browser (don't do this in production though!)
import ReactDOMServer from "react-dom/server";
import App from "./yourComponent";
document.body.innerHTML = ReactDOMServer.renderToStaticMarkup(App);
var pageHTML = window.document.getElementById('divToPDF').innerHTML;
let data = new Blob([pageHTML], {type: 'data:attachment/text,'});
let csvURL = window.URL.createObjectURL(data);
let tempLink = document.createElement('a');
tempLink.href = csvURL;
tempLink.setAttribute('download', 'Graph.html');
tempLink.click();
You can build your code and host it on github.io. The following tutorial will help you achieve that. You can then use the generated code in the gh-pages branch as your exported HTML
This was the first thread I found on SW.. so I think it would be appropriate to copy my own answer from another thread: https://stackoverflow.com/a/72422258/1215913
async function saveToFile() {
const handle = await showSaveFilePicker({
suggestedName: 'index.html',
types: [{
description: 'HTML',
accept: {'text/html': ['.html']},
}]
});
const writable = await handle.createWritable();
await writable.write(document.body.parentNode.innerHTML);
writable.close();
}; saveToFile();
for more info check the source answer
I had done this before but was stuck and couldn't seem to find the documentation anywhere. My scenario was I had a react js SPA and needed to create a static build to run without a server (through an organisations SharePoint using a static doc repository).
It is pretty simple in the end, run
npm run build
in your project directory and it will create the static build in a 'build' folder ready for you to dump wherever needed.
Reference: https://create-react-app.dev/docs/production-build/