We got this alert and it appears that the controller server was unable to transfer the file to our production server due to the file too large to be transferred to our Production Server/s:
*exception: tooltwist.fip.FipException: File is too large to be downloaded: tomcat/bin/synnexESDClient.2013-04-30.log*
Upon checking on the controller's image, the file's size is 84MB:
-rw-rw-r--. 1 controller controller **84M** Apr 30 23:59 synnexESDClient.2013-04-30.log
What is the maximum filesize per file can the FIP service handle to transfer from controller to production server/s? Or is there a config file for FIP service that we can check?
I'm not certain of the maximum size of fip file transfers, but it's probably a power of 2 (32mb, 64mb, etc).
In any case, the purpose of FIP (File Installation Protocol) is to incrementally deploy an application to production servers. Including large log files in the software distribution process is likely to jam up your website updating process, as it is installed to a dozen or more servers (especially when some are on the other side of the country).
First thing, you might want to consider whether you really want to deploy a log file from the Controller to production servers (what is the Controller doing that creates that log file, and why do you want it on production servers?).
If you really need to copy that file to production servers, I suggest you do it independently to the software and web files installation process. To do this, include the log file in the exclusion list for fip and then copy it by hand.
Related
I have a nestjs project in which I need to store the videos and save their paths in the mysql database my question is: how can I create a createReadStream of one of these videos knowing that the videos and the database will be in one server and the application in another server ??
fs.createReadStream() works just fine if you give it a full OS path. If you're having trouble with that, then we need to see what your "absolute path" is.
Usually, you would not store a full OS path in a database because that "hard wires" your implementation to a specific disk layout. Instead, you would usually store a relative path only in the database and then combine that with a base path that's part of the configuration of your app. That gives you more flexibility if you ever need to rearrange how things are stored on disk without having to rewrite every path in the database. Instead, you can just change the base path in your configuration to point to the new path.
For example, imagine you outgrow your current system disk and add a new faster and larger disk. When storing only relative paths in the database, you could shut-down your app, copy all the files over to the new disk, then adjust the base path in your app, restart your app and it would be up and running with the new location. If you had stored the absolute path in your database, you'd have to write a DB script tor rewrite every single path in the database to the new location.
can i host the database in one server and the application in anthor ? if yes how can i do that ??
The database can be wherever you want. But, the files themselves that you want to stream as a response to an http request will have to be accessible from the web server. If you want to use fs.createReadStream() as the source to stream them, then the files have to be accessible via an OS file path because fs.createReadStream() only works with a file path. If the files are stored elsewhere that doesn't have OS file path access to your web server (like say in a cloud service), you'd have to find some other way to read/stream them from your web server.
How can I create a createReadStream of one of these videos knowing that the videos and the database will be in one server and the application in another server
You can use fs.createReadStream() only if you have OS level file access to the other server (likely via some shared file system mechanism). For example, the files could be stored on some shared file server.
If you don't have OS level file access, then you will have to assess what type of access you do have and pick an appropriate mechanism for streaming the files from there. For example, if you have web access to the files (they are accessible via some URL), then you could either redirect the client to get the files directly from the other web server or you could stream them from that other server yourself using an http library that supports streaming such as got() or axios(). You could then pipe that http stream into your response - similar to what you would do with the stream from fs.createReadStream().
I want to know the exact difference between FTP Task and File System Task i gone through some tutorials but i didn't get any major differences
Can anyone help me out with this
They give the same functionality but use complete different mechanics:
File System Task is used to handle local or network (accessible, mapped drives) files. You can also change files properties to hide a file or make it read-only.
FTP Task uses the File Transfer Protocol to interact with an FTP server to move, create or delete files. This protocol involves a lot of communication procedures and handling to do these operations.
For an FTP communication you need a client and a server (that is a software running all the time listening to communications on one end), while in the File System Task the SSIS just interacts to the OS file system directly.
When creating a new project from existing files on a remote server (via SFTP), it takes a long time to get all the files (three hours for a Magento 2 installation). If I were to compress and transfer the installation, it would take less than ten minutes to transfer.
Is there a way I can manually transfer the files (outside of PhpStorm) onto my dev machine, then have PhpStorm index the local copy of the files for its code intelligence, but tell it about the link to the remote server, for deployment and debugging?
My site that heroku hosts takes user inputs and update a json file on a remote server. I should have probably stored in a database. It's a better solution. But is there any way I can download the most up to date Json files on the Heroku remote server?
Heroku doesn't offer any mechanism to commit files directly on the server, or to copy files from the server. One of the main reasons is its ephemeral filesystem:
Each dyno gets its own ephemeral filesystem, with a fresh copy of the most recently deployed code. During the dyno’s lifetime its running processes can use the filesystem as a temporary scratchpad, but no files that are written are visible to processes in any other dyno and any files written will be discarded the moment the dyno is stopped or restarted. For example, this occurs any time a dyno is replaced due to application deployment and approximately once a day as part of normal dyno management.
If the file is accessible over the web you might be able to download it from your browser, but whatever file you created may not be there anymore. You're right that a database is a better choice.
I have a VB6 application that works fine on most Windows 7 machines (even with UAC turned on), but for some of them if the program is not set to 'Run as administrator' upon startup it will return the error message 'No value given for one or more required parameters' when it tries to query the database.
I know the error message usually means that the table name(s) and/or parameter(s) are spelled incorrectly. But that is not the case here since the same application doing the same proceedures/query calls has no issues on Win XP and some Win 7 machines.
The database is MS Access 2003 format. The database is located in the Program Files directory along with the application exe and dlls.
If this was a consistant error then I could easily debug it and move on, but since all my testing machines do not generate this error, I am at a loss.
Any ideas as to why this occurs and how to fix it?
Thanks,
Chris
Storing database file in Program Files is not a good idea because this location is protected and standard users, and administrators in UAC-enabled system, cannot write to it. You should store the database either in AppData in user's profile or ProgramData if it needs to be shared between all users.
Since standard users cannot write to Program Files, Vista/7 have Virtualization mechanism. If a program without write access to Program Files tries to write there, the file system redirects the request to user's profile. The virtualized Program Files directory is located in C:\Users\<account>\AppData\Local\VirtualStore. I think you see this error because of virtualization: the database may exist in both locations the real Program Files and the virtualized one, and the files can be different. The virtualized version could not have the required records, that's why you get the error message.
When you start your application as Administrator, Virtualization is disabled and you access the file located in Program Files.
So check if the database file exists in VirtualStore, and try to find the differences between it and the file stored in Program Files.