I am using latest version of drill i.e (1.5 version) and using drill in embedded mode for local.
I have some csv file in my other system(PC2), which has some I.P address. I want to run search query from my own system(PC1) and try to get that csv file which is store in other system(PC2). PC1 have Drill and its running throught cmd in embedded mode.
Is There any way to get data or to search files(csv, psv, etc) from other pc(remote machine) in embedded mode for local(not for hdfs)?
You could try to mount drive from remote pc as local drive, and then run your drill queries on local drive. I did that with linux PC.
But this is not best decision for querying big files. Better to setup drill in distributed mode and run drillbit on machine where you store your data.
Related
I have a nestjs project in which I need to store the videos and save their paths in the mysql database my question is: how can I create a createReadStream of one of these videos knowing that the videos and the database will be in one server and the application in another server ??
fs.createReadStream() works just fine if you give it a full OS path. If you're having trouble with that, then we need to see what your "absolute path" is.
Usually, you would not store a full OS path in a database because that "hard wires" your implementation to a specific disk layout. Instead, you would usually store a relative path only in the database and then combine that with a base path that's part of the configuration of your app. That gives you more flexibility if you ever need to rearrange how things are stored on disk without having to rewrite every path in the database. Instead, you can just change the base path in your configuration to point to the new path.
For example, imagine you outgrow your current system disk and add a new faster and larger disk. When storing only relative paths in the database, you could shut-down your app, copy all the files over to the new disk, then adjust the base path in your app, restart your app and it would be up and running with the new location. If you had stored the absolute path in your database, you'd have to write a DB script tor rewrite every single path in the database to the new location.
can i host the database in one server and the application in anthor ? if yes how can i do that ??
The database can be wherever you want. But, the files themselves that you want to stream as a response to an http request will have to be accessible from the web server. If you want to use fs.createReadStream() as the source to stream them, then the files have to be accessible via an OS file path because fs.createReadStream() only works with a file path. If the files are stored elsewhere that doesn't have OS file path access to your web server (like say in a cloud service), you'd have to find some other way to read/stream them from your web server.
How can I create a createReadStream of one of these videos knowing that the videos and the database will be in one server and the application in another server
You can use fs.createReadStream() only if you have OS level file access to the other server (likely via some shared file system mechanism). For example, the files could be stored on some shared file server.
If you don't have OS level file access, then you will have to assess what type of access you do have and pick an appropriate mechanism for streaming the files from there. For example, if you have web access to the files (they are accessible via some URL), then you could either redirect the client to get the files directly from the other web server or you could stream them from that other server yourself using an http library that supports streaming such as got() or axios(). You could then pipe that http stream into your response - similar to what you would do with the stream from fs.createReadStream().
For my web database I am trying to create a webserver so I need to make a conf file. When I try to edit my conf file and save it wont let me and says permission denied. I have already given myself read and write access, but I still don't have access to the edit the file. Here is the link to the resource I am using to set up my webserver : https://github.com/orsenthil/adminer-on-mac
I've been working with local web servers and mysql on Mac for over 15 years, so this recommendation is coming from that experience.
OS X updates will destroy your local web development environment every time you do a major OS update... Don't use the built in Apache, PHP and MySQL. Use Homebrew to install independent web server components.
Now to your problems:
/etc/hosts is a file not a directory. Each line is a separate record. The line they gave 127.0.0.1 apache.local just means that your local computer will be accessible from http://apache.local. This hosts file can be used to avoid DNS lookups for any website.
The conf file you are referring to is the Apache VirtualHost file? OS X is very protective about files in etc. Have you tried sudo? If you change the permissions to your user, Apache may not be able to function.
An ASP.NET application (running on Windows server/IIS 7) has to transfer big size files uploaded by current user to an external SFTP server. Due to the file size the idea is to do this asynchronously.
The idea is that the ASP.NET application stores the uploaded file on a local directory of the Windows server. The current user can continue his work. A Windows service or a Quartz job (other tools(*)/ideas?) is now responsible to transfer the file to the external SFTP server.
(*) Are there existing tools that listen on changes of a Windows directory and then move the files on a SFTP server (incl. handling communication errors/retries)?
If there is no existing solution, do you have had similar requirements? What do we have to consider? Because the connection to the SFTP server is not very stable we need an optimized error handling with auto retry functionality.
To watch for changes in a local directory in .NET, use
the FileSystemWatcher class.
If you are looking for an out of the box solution, use the keepuptodate command in WinSCP scripting.
A simple example of WinSCP script (e.g. watch.txt):
open sftp://username:password#host/
keepuptodate c:\local_folder_to_watch /remote_folder
exit
Run the script like:
winscp.com /script=watch.txt
Though this works only, if the uploaded files are preserved in the remote folder.
(I'm the author of WinSCP)
My Django Application is running on "http://culhapartners.pythonanywhere.com/Front/Front_End"
Actually all this data for the website is in Django database on this online server.
My problem is "I want to run this website even if there is not internet connectivity"
how to move all database data to local pc directory?
Any suggestion as a starting point
Thank you
Regards
Noman
You can create a data dump with the dumpdata management command, then transfer this file to your local computer. Then you can load this data with the appropriately named loaddata command.
Or, if its a lightweight website and you are using sqlite, you can just copy the database file to your local machine.
Lots of conflicting reports out there when I google; can anyone help?
Basically I want to have the same databases available at work & home PCs, for development porpoises only.
I will only ever be physically at one PC (with a 45 minute trip between) and only then will there be database access. The MySql service will be running on both, but only one will be write/reading the database. Both run Windows 7
I don't need to bother with symlinks, junctions, etc as I have enough space in my DropBox to install all of Xampp there.
Can I do that without corruption?
I use something similar as part of my remote development testing, I have never tried to run a large set of data from this or a production database. Haha, but for testing queries and setting up a couple rows, etc... I use something very similar with little to no problems.
I believe that you can run the portable xampp and mysql from your Dropbox just go to www.portableapps.com to get the xampp and also download the portable launcher from the link on the page.
Let XAMPP server run on your computers, and the data folder saved on your cloud (DropBox, SkyDrive, etc).
You should install XAMPP portable on your root folder (c:) that is the same on all computer you use, then you can move XAMPP data folder (xampp Apache htdocs) and MySQL data (xampp Mysql/data) folder to Dropbox.
Do not use XAMPP installer, but use the portable one.
Configure the folder settings on both Apache and MySQL config file:
C:\xampp\apache\conf\httpd.conf
C:\xampp\apache\conf\extra\httpd-ssl.conf
C:\xampp\apache\conf\extra\httpd-xampp.conf
C:\xampp\mysql\bin\my.ini
Do the same with other server that you use (PostgreSQL, etc).
Make sure that you:
Always turn off XAMPP server after using on each PC.
Always sync Dropbox before changing PC, that is before logging on and after logging off on different computer.
Mind host / local domain variable on each computer.
Its probably better to use same username for all computer you want to use to run the data folder.
Other method is using PortableAppas.com, that is running the apps directly from USB.