I'm trying to get & store in a text file all the all addresses from the "address" column in the next page:
http://bitcoinrichlist.com/top100
It should be very simple, but I never worked with something like this before, I always received data from mysql databases or php variables, but never from a html page.
Someone may explain me how should I do this?
Storing the addresses it's easy, fwrite command right to a text file on my server.
But what about receiving the data? How should I do it?
edit:
My question in short is: How should I save all the addresses from the "address" column in a text file?
You can use the PHP Simple HTML DOM Parser to pull in a web page and then traverse its DOM. Example from their quick start guide:
// Create DOM from URL or file
$html = file_get_html('http://www.google.com/');
// Find all images
foreach($html->find('img') as $element)
echo $element->src . '<br>';
// Find all links
foreach($html->find('a') as $element)
echo $element->href . '<br>';
Once you gather up the elements, loop through them and write out to a file.
Related
I am looking to use a batch command to create a new text document in a specific folder.
#echo off
echo.>"[Path]\[DocName].txt"
This allows me to create a document in a specific folder. All thats working great.
Now i am wanting to pass variables/parameters into that text file. How can I pass a variable to populate in the text document i just saved?
additional question, can I do this in a CSV instead of a txt? I only seem to be able to create a txt document.
To add several lines of data to a new text file, from a batch-file, this is the recommended syntax:
#(
Echo(%1
Echo Something before an empty line
Echo(
Echo Something after an empty line
)>"[Path]\[DocName].txt"
The most important thing to remember, when using this methodology is that any ) characters to be Echoed must be escaped using a caret, i.e. Echo This works, (hopefully^).
echo text > file.txt
Change the text to whatever you want. Change the file name to whatever you want (It doesn't have to be a .txt file, you can save it as a .csv as you wanted.) If you want to pass a variable into the file,
echo %var% > file.txt
Easy as that.
I'm a beginner in YII and I don't know whats the difference between <?= Html::encode($this->title)?> to just <?= $this->title?>.. I have tested these two and they seemed to give the same output.
Docs and source code can tell you everything.
Basically Html::encode() is just the wrapper of htmlspecialchars native PHP function:
Encodes special characters into HTML entities.
The application charset will be used for encoding.
To understand the basic difference and benefit of that, try to echo:
$string = '<script>alert(1);</script>';
echo $string;
and then:
echo Html::encode($string);
So encode is useful for filtering user saved data. If the data comes from developer, not from user, you may not apply encode and HTML will be displayed as is.
Official docs:
Html::encode()
htmlspecialchars
In http://www.githubarchive.org/ that Ilya Grigorik has provided ,I found that in many gz files , some consecutive events are logged to same file .
for example in 2011-03-15-21.json.gz
To get the above do :
wget http://data.githubarchive.org/2011-03-15-21.json.gz
In this gz for example if you search for id 1484832 , you can find that the 2 consecutive events(jsons) are in same line
see
http://codebeautify.org/jsonviewer/2cb891
the two jsons in same line is a combination of
http://codebeautify.org/jsonviewer/c7e18e
and
http://codebeautify.org/jsonviewer/945d56
.
What is the impact ?
when I was loading each line and loading it with python's(why python ? because I felt python is comfortable in dealing with jsons) json.loads it said it was invalid as it was a combination of two jsons .
Question :
1) How did you solve these kind of bugs when you processed that github archive data ?
2) I already have the data in my local . so how can I overcome this problem . Shall I write code specific to this case to overcome ?
the code i wrote was like
jsonlist = line.split('}{')
json.loads(jsonlist[0] + '}', "ISO-8859-1") # load and navigate through this json
json.loads('{' + jsonlist[1], "ISO-8859-1") # load and navigate through this json
I got the solution here
1) How did you solve these kind of bugs when you processed that github archive data ?
https://github.com/vadasg/githubarchive-parser/blob/master/src/FixGitHubArchiveDelimiters.rb
. This script removes the problems of two or more events appearing on the same line .
so now after running this script the jsons appear in different lines .
2) I already have the data in my local . so how can I overcome this problem . Shall I write code specific to this case to overcome ? the code i wrote was like
This script removes the necessity to write the code I mentioned above .
Note :
Related issues found on the github archive project in github
https://github.com/igrigorik/githubarchive.org/issues/53
https://github.com/igrigorik/githubarchive.org/issues/17
WARNING :
When I was running this script I got an error related to the encoding used . Because by default the Yajl::Parser.parse(jsonInputFile)
line checks if characters it parses adheres to UTF-8 encoding ,if not it will throw errors .
As github data also contains non UTF-8 characters , this error will be thrown in our case too. So to bypass that problem(or may be a fix) I put it as
Yajl::Parser.parse(jsonInputFile, :check_utf8 => false)
for doubts refer docs: http://rdoc.info/github/brianmario/yajl-ruby/Yajl/Parser.parse
How can I get all meaning of a word in json response .Preferably without key/signing up/api key .If I type test then all meaning of that word.
I tried :
but its not working.
<?php
$query = 'oracle';
$file = file_get_contents('http://www.google.com/dictionary/json?callback=a&q='.$query.'&sl=en&tl=en&restrict=pr,de&client=te');
// var_dump($file);
$file = substr($file, 2, -10);
$file = preg_replace("/\\\x[0-9a-f]{2}/", "", $file);
echo $file;
$json = json_decode($file);
var_dump($json);
?>
Even this is returning null.
I have tried Only the php above.I would like to knowif I can make rest call without api key just words which match the query word .Is there any rest call you have in mind.I really appreciate any help .Thanks in Advance.
Assuming server side
Get a copy of a dictionary in a computer friendly format i.e. http://www.ibiblio.org/webster/ (XML).
Store said dictionary in a database or in memory and perform a lookup.
Would then be trivial to provide a restful service returning all definitions for a particular word.
Also see: Google's "define: " through an API?
I am trying to insert image in database. The database has a field named images and its type is BLOB. I am trying to insert the image, but only the first 2.2KB is stored. Even if I insert another image it stores only 2.2KB in the database.
When I try to show this image in my application it doesn't show; it's just a small icon, not the image. How can I insert the image in the right way?
use CGI;
my $file = $q->param("file");
$file = 'C:/wamp/bin/apache/apache2.2.22/cgi-bin/images/2.jpg';
open(my $fh, $file);
my $data;
binmode($fh);
read($fh, $data, (stat($fh))[7]);
close($fh);
my $Data = {
table =>'student',
data => {
Image => $fh,
}
};
Data::Insert($Data);
print $q->header;
print $q->start_html(
-title => "student",
);
print $q->end_html;
showImage.pl
my $q = new CGI();
my $handle = Dbm::connection();
$id = $q->param('id_person');
$getimage = $handle->selectrow_array (<<SQLEOF);
SELECT Image
FROM student
WHERE ID = '$id'
SQLEOF
print "Content-Type: image/jpeg\n";
print "Content-length: \n\n";
binmode STDOUT;
print STDOUT $getimage;
My recomendation is keep the image as base 64 encrypted to the db with the MIME type of the image. When you need it, just decrypt it by saying MIME type. This is the mostly used to upload files using ajax. So why can't we use the same way to store image directly to DB?
Just give an additional column to keep MIME type in your table and take it along with the encoded data as print it together.
From a file extension, we can identify the type of file. The MIME type for images are mainly
image/gif: GIF image
image/jpeg: JPEG JFIF image;
image/pjpeg: JPEG JFIF image;
image/png: Portable Network Graphics;
image/svg+xml: SVG vector image;
image/tiff: Tag Image File Format (only for Baseline TIFF);
You can create a new column by giving name as mime_typ . Now when you enctrypt a file using base 64 encryption, keep it as a string like we store usernames and passwords in a table. Similarly add the MIME type to the mime_typ column. when you want to show the image, print the encrypted content after decoding it along with the content in the MIME type, which is stored in the same row in the mime_typ column. You can search google for the way to show an image which is encrypted in base 64 encryption.
You need to read the file in binary mode - i.e.
open(my $fh, $myfile);
my $data;
binmode($fh);
read($fh, $data, (stat($fh))[7]);
close($fh);
I'm not sure why you deleted the database-insert code from the question, but I found it in the revision history.
The issue could be because you aren't not using bind variables, and the binary image contains an escape character which is causing a problem.
I recommend using DBIx::Simple to help create your insert statements that will help create the bind variables for you. DBIx::Simple works with both SQL::Abstract and SQL::Interp. I find SQL::Interp more flexible.
It also appears to be a bug that you are inserting the file handle into the image field instead of the file data. Try adding use File::Slurp (which you may need to install), and then putting this in your %data hash:
Image => scalar read_file($file, { binmode => ':raw' });
Your SELECT statement is also vulnerable to a SQL injection attack because you did not validate the outside input before passing it to the database, and you did not you bind variables again. Using DBIx::Simple, the same code would look like this:
my $db = DBIx::Simple->new($handle);
$getimage = $db->iquery("SELECT Image FROM student WHERE ID = ",\$id)->list;
Also, I recommend omitting the Content-Length header, or properly calculating it, rather than leaving it present in an invalid state.