PHP Phantomjs: How to handle Page Transition? (Web Scraping) - unique

Everyone.
This question maybe duplicated with Using PhantomJs, how to get and handle the new page?.
But there was not exact answer.
My question is as follows.
The pages what I want to scrape 3 pages.
1st Page : input unique id , click next.
if valid id => go to 2nd page.
2nd Page : click link(contains id)
3rd Page : download Pdf File.
So my aim is to download pdf file from unique id automatically.
Then the main point is how to handle page transition in Phantom PHP?
My test Code is as:
// Use the composer autoloader.
require_once 'vendor/autoload.php';
// Setup mink drivers.
$goutteDriver = new \Behat\Mink\Driver\GoutteDriver();
$phantomjsDriver = new \Behat\Mink\Driver\Selenium2Driver('phantomJS');
// Setup mink sessions.
$goutteSession = new \Behat\Mink\Session($goutteDriver);
$phantomjsSession = new \Behat\Mink\Session($phantomjsDriver);
// Setup mink session manager.
$mink = new \Behat\Mink\Mink();
// Register sessions.
$mink->registerSession('goutte', $goutteSession);
$mink->registerSession('phantomjs', $phantomjsSession);
// Set Goutte as the default session.
$mink->setDefaultSessionName('phantomjs');
// Visit mink website with phantomjs driver.
$mink->getSession('phantomjs')->visit('https://testurl.com');
// Get the default goutte session.
$session = $mink->getSession('phantomjs');
// Get the page document.
$page = $session->getPage();
echo $session->getCurrentUrl(), PHP_EOL;
// $page->find('css', '#guides')->clickLink("Drivers");
// echo $session->getCurrentUrl(), PHP_EOL;
// Output the installation instructions from the page.
$input = $page->find('css', '#id');
$input->setValue("1234567890");
echo $input->getValue(), PHP_EOL;
$page->find('css', '#validar')->Click();
echo $page->find('css', '#validar')->getValue(), PHP_EOL;
$session->executeScript('document.getElementById("validar").click()');
//$session->reload();
sleep(5);
// $mink->getSession('phantomjs')->visit('https://testurl.com/next');
$page = $session->getPage();
echo $session->getCurrentUrl(), PHP_EOL;
sleep(5);
$mink->getSession('phantomjs')->visit('https://testurl.com/next2?id=1234567890');
$page = $session->getPage();
echo $session->getCurrentUrl(), PHP_EOL;
// Stop browser sessions.
$mink->stopSessions();
So
How to handle page transition?
How to download pdf file properly?

Related

HTML5 - Server side events - Using external text file as data input

I am attempting of using a plain text-file (text.txt) as part of html5 - server side events.
Currently I am not seeing any printout of text in browser. If just running this line, without using an external file for input data, it works:
echo "data: test\n\n";
Question: What do I need to adjust to make the external file data to be visible in the browser, assuming the setup of below files?
My html file
<h1>SSE</h1>
<div id="result"></div>
<script>
// Create an object
var source = new EventSource("updater.php");
// Detect message receipt
source.onmessage = function(event) {
// Write the received data to the page
document.getElementById("result").innerHTML += event.data + "<br>";
};
</script>
my php file
<?php
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache');
// Include files
echo include("text.txt");
flush();
?>
my text file:
"data: t1972\n\n";
The solution for the question, works if using [file_get_contents] as specified below.
$data = file_get_contents('text.txt');
echo "data: " . $data . "\n\n";
This also means that the file [text.txt] is left with only text, removing the [data: ] and [\n\n].

Connect to a second database from wordpress

I am trying to connect to my own self created second database using wordpress.
What i am trying to achieve is to extract data from my own database(not wordpress database) on my static wordpress front page.
I use php_everywhere plug in in order for me to write php code in blog post and pages.
I use the following code:
<?php
$servername = "localhost";
$username = "thomas";
$password = "password";
$dbname = "thomas";
// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$sql = "SELECT * from number";
$result = $conn->query($sql);
if ($result->num_rows > 0) {
// output data of each row
while($row = $result->fetch_assoc()) {
echo "id: " . $row["id"]. " - Name: " . $row["firstname"]. " " . $row["lastname"]. "<br>";
}
} else {
echo "0 results";
}
$conn->close();
?>
However i got an error unexpected '"localhost"' (T_CONSTANT_ENCAPSED_STRING).
Not sure why.
See https://stackoverflow.com/search?q=T_CONSTANT_ENCAPSED_STRING
1) When using that plugin, be sure you've used straight quotes and backspace around the php to remove invisible control characters. And be sure the plugin doesn't have limitations on what it can do and ignores paragraph returns in the editor.
2) You're much better off using a page template in your theme and running PHP in a .php file and not in the post/page editor with a plugin. See Page Templates | Theme Developer Handbook | WordPress Developer Resources. Make a page template, add the calls to header, etc., and then work with your PHP in the file itself. Select the page template in the page editor of the page you're publishing in WordPress.
You can create new object for new database connection and use that object to do any operation for external database
$mydb = new wpdb('thomas','password','thomas','localhost');
$rows = $mydb->get_results("select Name from my_table");
You must have to use $mydb in place of $wpdb
Wordpres provide a way to connect the other db.
$mydb = new wpdb('thomas','password','thomas','localhost');
$rows = $mydb->get_results("select Name from number");
foreach ($rows as $row) :
echo "id: " . $row->id. " - Name: " . $row->firstname. " " . $row->lastname. "<br>";
endforeach;

How can I embed external content in a WordPress widget?

I want to display a download link inside a WordPress widget. The file to be downloaded is located in the download subfolder of the site root, so that it can be uploaded via FTP. The name of the file and the text to be displayed for the download link shall be stored in a simple text file in the same folder.
Assuming WordPress is installed on www.mysite.com. The file name is setup_1_0.zip and the link display is Setup 1.0.
I am open to the file format how this information is stored as long as I can upload that file via FTP, too.
How can I embed this information inside a Custom HTML widget to get a valid download link with the text taken from that file?
How to automate the process of uploading latest software's build and download link creation in WordPress?
Based on your logic.
You are trying to automate the download process of your latest software version.
You don't want to update things manually and you just want to upload your latest build in the /download/ folder. (Only drop your latest version using FTP; that's all)
This is how I would do it:
Referencing those questions:
Get the latest file addition in a directory
How to force file download with PHP
I propose two solutions: First two separte codes, Second One inline code.
Just for educational purpose
First solution: Quick and short usage:
(You might need a way or a plugin to activate running PHP in Widget; this plugin helps PHP Code Widget)
<?php
$path = "download/";
$latest_ctime = 0;
$latest_filename = '';
$d = dir($path);
while (false !== ($entry = $d->read())) {
$filepath = "{$path}/{$entry}";
// could do also other checks than just checking whether the entry is a file
if (is_file($filepath) && filectime($filepath) > $latest_ctime) {
$latest_ctime = filectime($filepath);
$latest_filename = $entry;
}
}
echo 'Download '. $latest_filename . '';
?>
Second solution:
(Again, you might need a way or a plugin to activate running PHP in Widget; this plugin helps PHP Code Widget)
A) Create download.php in http://www.example.com/download.php
Add the following code:
<?php
$path = "download";
$latest_ctime = 0; //ctime stands for creation time.
$latest_filename = '';
$d = dir($path);
while (false !== ($entry = $d->read())) {
$filepath = "{$path}/{$entry}";
// could do also other checks than just checking whether the entry is a file
if (is_file($filepath) && filectime($filepath) > $latest_ctime) {
$latest_ctime = filectime($filepath);
$latest_filename = $entry;
}
}
// echo $latest_filename; un-comment to debug
$file_url = 'http://www.example.com/download/'.$latest_filename;
header('Content-Type: application/octet-stream');
header("Content-Transfer-Encoding: Binary");
header("Content-disposition: attachment; filename=\"" . basename($file_url) . "\"");
readfile($file_url); // do the double-download-dance (dirty but worky)
?>
B) in your WordPress HTML Widget add the following code
<?php
$path = "download";
$latest_ctime = 0;
$latest_filename = '';
$d = dir($path);
while (false !== ($entry = $d->read())) {
$filepath = "{$path}/{$entry}";
// could do also other checks than just checking whether the entry is a file
if (is_file($filepath) && filectime($filepath) > $latest_ctime) {
$latest_ctime = filectime($filepath);
$latest_filename = $entry;
}
}
echo 'Download '. $latest_filename . '';
?>
Further explanation:
A) is responsiple for downloading the latest software build automatically.
B) is responsiple for displaying Latest build name and Creating the link.
Now, You only need to upload one file to your /download/ folder which is your latest build (setup_1_0.zip, setup_1_1.zip, setup_1_2.zip ...etc. The proposed solution will check creation date regardless of file's name.)
Important note: You can see that the latest file checker function is repeated twice; once in download.php and once in WordPress Widget. Because if we combine in one file we will get header already sent error.
Dose this answer your question please? Kindly feedback.

Why is code loading before doctype

<?php
# Alert the user that this is not a valid access point to MediaWiki if they try to access the special pages file directly.
if ( !defined( 'MEDIAWIKI' ) ) {
echo <<<EOT
To install my extension, put the following line in LocalSettings.php:
require_once "$IP/extensions/Userprofile/Userprofile.php";
EOT;
exit( 1 );
}
$wgExtensionCredits['specialpage'][] = array(
'path' => __FILE__,
'name' => 'Userprofile',
'author' => 'matsuiny2004',
'url' => 'http://localhost/mywiki/index.php/Extension:Userprofile',
'descriptionmsg' => 'userprofile-desc',
'version' => '0.0.0',
);
$wgAutoloadClasses['SpecialUserprofile'] = __DIR__ . '/SpecialUserprofile.php'; # Location of the SpecialMyExtension class (Tell MediaWiki to load this file)
$wgMessagesDirs['Userprofile'] = __DIR__ . "/i18n"; # Location of localisation files (Tell MediaWiki to load them)
$wgExtensionMessagesFiles['UserprofileAlias'] = __DIR__ . '/Userprofile.alias.php'; # Location of an aliases file (Tell MediaWiki to load it)
$wgSpecialPages['Userprofile'] = 'SpecialUserprofile'; # Tell MediaWiki about the new special page and its class name
function extensionFunction() {
# Assume $title is the title object
if( $title->isProtected( 'edit' ) ) {
# Protected from editing, do things
} else {
# Not protected from editing
}
}
//test code here
echo '<div id="navigation">Navigation</div>';
?>
<?php
echo '<div id="account">account</div>';
?>
<?php
echo '<div id="editpage">edit page</div>';
?>
<div id='border-search'>
<img src="http://s6.postimage.org/z6ixulv6l/searchbox_border.png"></img>
</div>
?php>
<div class='rectangle-box'>
<div class='rectangle-content'></div>
</div>
I am including extra php and html code from a seperate file that I load as an extension in mediawiki to create a custom layout. the problem is that the code loads before the doctype making the page render in quirks mode in IE and safari. How can i get it to load after the doctype tag?
The problem is that you have top-level echo statements all over your code (by top level I mean ones that are not included in any function). Which is why the PHP engine executes them as soon as it sees them, which is before the MediaWiki itself starts running.
MediaWiki has some good documentation explaining all kinds of hooks. What you should do is write a few functions and place all your echo-ing code there, and expect this function to be executed when the event in question occurs.
I see a good starter page here. Relevant example is:
$wgHooks['ArticleSave'][] = 'wgAddStub';
function wgAddStub( &$article, &$user, &$text, &$summary, $minor, $watchthis, $sectionanchor, &$flags, &$status ) {
$text = ( $article->exists() ? "" : "{{stub}}\n" ) . $text;
return true;
}

AS3 ALIVEPDF saving via method.remote (PHP) no longer working

The SWF is located on a web server. I am calling the function using this code in AS3...
myPDF.save(Method.REMOTE, "http://www.example.com/generator/createpdf.php",
Download.ATTACHMENT, "line.pdf");
Here is my PHP script located on the server...
$method = $_GET['method'];
$name = $_GET['name'];
if ( isset ( $GLOBALS["HTTP_RAW_POST_DATA"] )) {
// get bytearray
$pdf = $GLOBALS["HTTP_RAW_POST_DATA"];
// add headers for download dialog-box
header('Content-Type: application/pdf');
header('Content-Length: '.strlen($pdf));
header('Content-disposition:'.$method.'; filename="'.$name.'"');
echo $pdf;
} else echo 'An error occured.';
It used to work, but stopped a while back. Any help would be greatly appreciated.
1) This stopped working for me as well, until I added the following -
if(!$HTTP_RAW_POST_DATA){
$HTTP_RAW_POST_DATA = file_get_contents(‘php://input’);
}
2) I also patched /src/org/alivepdf/pdf/PDF.as::save() per this post enter link description here