I am working on a project using meteor and collectionfs.
I upload files to collectionfs and have a filehandler in place. I can use the
{{cfsFileUrl "defaultFilehandler"}}
Handlebar Helper to display the url where the image is saved, but I can't download images from this URL.
When I copy it into my browser:
localhost:3000/cfs/contacts/Nj3WzrBKhqd9Mc9NP_defaultHandler.png
meteor routes me to the meteor page ( as if i had writen localhost:3000 )
Ultimately I would like to achieve two things:
1st
display the image using an html tag:
<img src=??? alt="your image" />
2nd
I would like to make sure that the user is allowed to see this image.
Having the 'download-url' is not sufficient security for me.
In order to get to the point i went through the normal tutorial from collectionFS:
client js
ContactsFS = new CollectionFS('contacts', { autopublish: false });
Deps.autorun(function() {
Meteor.subscribe('myContactsFiles');
});
Template.queueControl.events({
'change .fileUploader': function (e) {
var files = e.target.files;
for (var i = 0, f; f = files[i]; i++) {
ContactsFS.storeFile(f);
}
}
});
server js
ContactsFS = new CollectionFS('contacts', { autopublish: false });
ContactsFS.allow({
insert: function(userId, file) {
console.log('user'+userId+"file"+JSON.stringify(file));
console.log("WILL SAVE:"+userId && file.owner === userId );
return userId && file.owner === userId;
},
update: function(userId, files, fields, modifier) {
return _.all(files, function (file) {
return (userId == file.owner);
}); //EO iterate through files
},
remove: function(userId, files) { return false; }
});
Meteor.publish('myContactsFiles', function() {
if (this.userId) {
return ContactsFS.find({ owner: this.userId }, { limit: 30 });
}
});
ContactsFS.fileHandlers({
default1: function(options) { // Options contains blob and fileRecord — same is expected in return if should be saved on filesytem, can be modified
return { blob: options.blob, fileRecord: options.fileRecord }; // if no blob then save result in fileHandle (added createdAt)
}});
Answer 1:
You can use the updated 0.3.3+ version of collectionFS to publicly make those files accessable, or view them.
<img src="{{cfsFileUrl 'default1'}}">
where default1 is the name of the handler function you defined in ContactsFS.fileHandlers
Answer 2:
There is no secure solution build into collectionFS as of now, its in the making tho.
Related
This question already has an answer here:
How can I resize all existing images in firebase storage?
(1 answer)
Closed 9 months ago.
I have requirement to resize new and existing images stored in firebase store. For new image, I enabled firebase's resize image extension. For existing image, how can I resized the image and get the newly resized image url to update back to database via api.
Here is my firebase function to get existing image urls from database. My question is how to resize the image and get the new image url?
const functions = require("firebase-functions");
const axios =require("axios");
async function getAlbums() {
const endpoint = "https://api.mydomain.com/graphql";
const headers = {
"content-type": "application/json",
};
const graphqlQuery = {
"query": `query Albums {
albums {
id
album_cover
}
}`
};
functions.logger.info("Call API");
const response = await axios({
url: endpoint,
method: 'post',
headers: headers,
data: graphqlQuery
});
if(response.errors) {
functions.logger.info("API ERROR : ", response.errors) // errors if any
} else {
return response.data.data.albums;
}
}
exports.manualGenerateResizedImage = functions.https.onRequest(async () => {
const albums = await getAlbums();
functions.logger.info("No. of Album : ", albums.length);
});
I think the below answer from Renaud Tarnec will definitely help you.
If you look at the code of the "Resize Images" extension, you will see that the Cloud Function that underlies the extension is triggered by a onFinalize event, which means:
When a new object (or a new generation of an existing object) is
successfully created in the bucket. This includes copying or rewriting
an existing object.
So, without rewriting/regenerating the existing images the Extension will not be triggered.
However, you could easily write your own Cloud Function that does the same thing but is triggered, for example, by a call to a specific URL (HTTPS cloud Function) or by creating a new document in a temporary Firestore Collection (background triggered CF).
This Cloud Function would execute the following steps:
Get all the files of your bucket, see the getFiles() method of the
Google Cloud Storage Node.js Client API. This method returns a
GetFilesResponse object which is an Array of File instances.
By looping over the array, for each file, check if the file has a
corresponding resized image in the bucket (depending on the way you
configured the Extension, the resized images may be in a specific
folder)
If a file does not have a corresponding resized image, execute the
same business logic of the Extension Cloud Function for this File.
There is an official Cloud Function sample which shows how to create a Cloud Storage triggered Firebase Function that will create resized thumbnails from uploaded images and upload them to the database URL, (see the last lines of index.js file)
Note : If you have a lot of files to treat, you should most probably work by batch, since there is a limit of 9 minutes for Cloud Function execution. Also, depending on the number of images to treat, you may need to increase the timeout value and/or the allocated memory of your Cloud Function, see https://firebase.google.com/docs/functions/manage-functions#set_timeout_and_memory_allocation
In case someone need it. This is how I resized existing image.
const functions = require("firebase-functions");
const axios = require("axios");
const { Storage } = require("#google-cloud/storage");
const storage = new Storage();
// Don't forget to replace with your bucket name
const bucket = storage.bucket("projectid.appspot.com");
async function getAlbums() {
const endpoint = "https://api.mydomain.com/graphql";
const headers = {
"content-type": "application/json",
};
const graphqlQuery = {
query: `query Albums {
albums {
id
album_cover
}
}`,
};
const response = await axios({
url: endpoint,
method: "post",
headers: headers,
data: graphqlQuery,
});
if (response.errors) {
functions.logger.error("API ERROR : ", response.errors); // errors
if any
} else {
return response.data.data.albums;
}
}
function getFileName(url) {
var decodeURI = decodeURIComponent(url);
var index = decodeURI.lastIndexOf("/") + 1;
var filenameWithParam = decodeURI.substr(index);
index = filenameWithParam.lastIndexOf("?");
var filename = filenameWithParam.substr(0, index);
return filename;
}
function getFileNameFromFirestore(url) {
var index = url.lastIndexOf("/") + 1;
var filename = url.substr(index);
return filename;
}
const triggerBucketEvent = async () => {
bucket.getFiles(
{
prefix: "images/albums", // you can add a path prefix
autoPaginate: false,
},
async (err, files) => {
if (err) {
functions.logger.error(err);
return;
}
const albums = await getAlbums();
await Promise.all(
files.map((file) => {
var fileName = getFileNameFromFirestore(file.name);
var result = albums.find((obj) => {
return getFileName(obj.album_cover) === fileName;
});
if (result) {
var file_ext = fileName.substr(
(Math.max(0, fileName.lastIndexOf(".")) || Infinity) + 1
);
var newFileName = result.id + "." + file_ext;
// Copy each file on thumbs directory with the different name
file.copy("images/albums/" + newFileName);
} else {
functions.logger.info(file.name, " not found in album list!");
}
})
);
}
);
};
exports.manualGenerateResizedImage = functions.https.onRequest(async () => {
await triggerBucketEvent();
});
I created a script that extracts photos in the gallery of a certain profile…
Using instagram-web-api
Unfortunately now it no longer works, instagram does not return the image of the media
This is the mistake:
ERR_BLOCKED_BY_RESPONSE
Instagram has changed it’s CORS policy recently? How I can fix?
for php; I changed my img src to this and it works like charm! Assume that $image is the instagram image cdn link came from instagram page:
'data:image/jpg;base64,'.base64_encode(file_get_contents($image))
EDIT FOR BETTER SOLUTION
I have also noticed that, this method is causing so much latency. So I have changed my approach and now using a proxy php file (also mentioned on somewhere on stackoverflow but I don't remember where it is)
This is my common proxy file content:
<?php
function ends_with( $haystack, $needle ) {
return substr($haystack, -strlen($needle))===$needle;
}
if (!in_array(ini_get('allow_url_fopen'), [1, 'on', 'true'])) {
die('PHP configuration change is required for image proxy: allow_url_fopen setting must be enabled!');
}
$url = isset($_GET['url']) ? $_GET['url'] : null;
if (!$url || substr($url, 0, 4) != 'http') {
die('Please, provide correct URL');
}
$parsed = parse_url($url);
if ((!ends_with($parsed['host'], 'cdninstagram.com') && !ends_with($parsed['host'], 'fbcdn.net')) || !ends_with($parsed['path'], 'jpg')) {
die('Please, provide correct URL');
}
// instagram only has jpeg images for now..
header("Content-type: image/jpeg");
readfile( $url );
?>
Then I have just converted all my instagram image links to this (also don't forget to use urlencode function on image links):
./proxyFile.php?url=https://www.....
It worked like charm and there is no latency anymore.
now 100% working.
You can try this.
corsDown
Using the Google translation vulnerability, it can display any image URL, with or without permission. All these processes are done by the visitor's IP and computer.
I have the same problem, when I try to load a Instagram's pictures url (I tried with 3 IP addresses), I see this on the console:
Failed to load resource: net::ERR_BLOCKED_BY_RESPONSE
You can see it here, the Instagram image doesn't load (Actually, when I paste this url on google it works, but Instagram puts a timestamp on there pictures so, it's possible it won't work for you).
It's very recent, 3 days ago, it works with no issues.
<img src="https://scontent-cdt1-1.cdninstagram.com/v/t51.2885-19/s320x320/176283370_363930668352575_6367243109377325650_n.jpg?tp=1&_nc_ht=scontent-cdt1-1.cdninstagram.com&_nc_ohc=nC7FG1NNChYAX8wSL7_&edm=ABfd0MgBAAAA&ccb=7-4&oh=696d56547f87894c64f26613c9e44369&oe=60AF5A34&_nc_sid=7bff83">
The answer is as follows. You can use the imgproxy.php file. You can do it like this:
echo '<a href="' . $item->link . '" class="image" target="_blank">
<span style="background-image:url(imgproxy.php?url=' . urlencode($thumbnail) . ');"> </span>
</a>';
Using PHP
u can grab content of the image and show it in php file as an image by setting the header:
<?php
$img_ctn = file_get_contents("https://scontent-ber1-1.cdninstagram.com/v/......");
header('Content-type: image/png');
echo $img_ctn;
You can display the Image using Base64 encoded.
Base64 func based on #abubakar-ahmad answer.
JavaScript:
export const checkUserNameAndImage = (userName) => {
/* CALL THE API */
return new Promise((resolve, reject) => {
fetch(`/instagram`, {
method: "POST",
headers: {
Accept: "application/json",
"Content-Type": "application/json",
},
body: JSON.stringify({ userName }),
})
.then(function (response) {
return response.text();
})
/* GET RES */
.then(function (data) {
const dataObject = JSON.parse(data);
/* CALL BASE64 FUCNTION */
toDataUrl(dataObject.pic, function (myBase64) {
/* INSERT TO THE OBEJECT BASE64 PROPERTY */
dataObject.picBase64 = myBase64;
/* RETURN THE OBJECT */
resolve(dataObject);
});
})
.catch(function (err) {
reject(err);
});
});
};
Base64 func:
function toDataUrl(url, callback) {
var xhr = new XMLHttpRequest();
xhr.onload = function () {
var reader = new FileReader();
reader.onloadend = function () {
callback(reader.result);
};
reader.readAsDataURL(xhr.response);
};
xhr.open("GET", url);
xhr.responseType = "blob";
xhr.send();
}
Now, instead of using the original URL, use the picBase64 property:
<image src={data.picBase64)}/>
I have built a simple PHP based media proxy to minimize copy&paste.
https://github.com/skmachine/instagram-php-scraper#media-proxy-solving-cors-issue-neterr_blocked_by_response
Create mediaproxy.php file in web server public folder and pass instagram image urls to it.
<?php
use InstagramScraper\MediaProxy;
// use allowedReferersRegex to restrict other websites hotlinking images from your website
$proxy = new MediaProxy(['allowedReferersRegex' => "/(yourwebsite\.com|anotherallowedwebsite\.com)$/"]);
$proxy->handle($_GET, $_SERVER);
I was too lazy to do the suggested solutions and since i had a nodejs server sending me urls i just wrote new functions to get the images, convered them to base64 and sent them to my frontend. Yes it's slower and heavier but it gets the job done for me since i don't have a huge need for performance.
Fetch and return base64 from url snippet
const getBase64Image = async (url) => {
return new Promise((resolve, reject) => {
// Safety net so the entire up
// doesn't fucking crash
if (!url) {
resolve(null);
}
https
.get(url, (resp) => {
resp.setEncoding("base64");
body = "data:" + resp.headers["content-type"] + ";base64,";
resp.on("data", (data) => {
body += data;
});
resp.on("end", () => {
resolve(body);
});
})
.on("error", (e) => {
reject(e.message);
});
});
};
You don't need any external modules for this.
I am trying to implement zip file upload functionality in Angular 8 app. 3 conditions that I need to satisfy are:
1. Only allow zip files to be uploaded else throw error message
2. File size should not cross 3 MBs else throw error message
3. When I choose zip file, it should show progress bar but file should only be uploaded via REST API call when I click 'Register' button separately.
What I have implemented so far is:File Upload Service
postFile(fileToUpload: File, header): Observable<any> {
const endpoint = 'your-destination-url';
const formData: FormData = new FormData();
formData.append('fileKey', fileToUpload, fileToUpload.name);
if (fileToUpload.size <= 3048576)
return this.httpClient.post(endpoint, formData, { headers: header })
.pipe(map(data => {
console.log(data);
return data;
},error => {
console.log(error, 'reduce file size');
}))
}
Component TS File
handleFileInput(files: FileList) {
this.fileToUpload = files.item(0);
}
uploadFileToActivity() {
this.fileUploadService.postFile(this.fileToUpload, this.headers).subscribe(data => {
// do something, if upload success
console.log('the file has been uploaded successfully', data);
}, error => {
console.log(error);
});
}
Component HTML
<input type="file"
id="file" (change)="handleFileInput($event.target.files)">
Please suggest how can I modify so that my functionality is as described.
for points 1 and 2 you should add a validation function in your code to check both the file extension and the size.
The upload should be possible only if the file passes the validation.
In addition to that, you should probably return some kind of feedback to the user when the validation fails.
You can track the file upload progress (and show a progress bar) adding additional options to the .post method and listening for specific events
return this.httpClient.post(endpoint, formData, {
headers: header,
reportProgress: true,
observe: 'events'
}).pipe(map(event => {
if (event.type === HttpEventType.Response) {
// upload complete
}
if (event.type === HttpEventType.UploadProgress) {
// the event contains information about loaded data
// you can use event.loaded and event.total to display the progress bar
}
}))
I am trying to figure out how to parse the JSON data that I am getting from the Spotify API. I am using this node module https://www.npmjs.com/package/spotify-web-api-js to get Spotify playlist tracks.
I am using this to GET my json (see what I did there)
export class HomePage {
spotifyApi = new SpotifyWebApi;
constructor() {}
}
var spotifyApi = new SpotifyWebApi();
spotifyApi.setAccessToken('Spotify OAuth Token');
spotifyApi.getPlaylistTracks('37i9dQZEVXbMDoHDwVN2tF')
.then(function(data) {
console.log('Playlist Tracks', data);
}, function(err) {
console.error(err);
var prev = null;
function onUserInput(queryTerm) {
// abort previous request, if any
if (prev !== null) {
prev.abort();
}
// store the current promise in case we need to abort it
prev = spotifyApi.searchTracks(queryTerm, {limit: 5});
prev.then(function(data) {
// clean the promise so it doesn't call abort
prev = null;
// ...render list of search results...
}, function(err) {
console.error(err);
});
}
This returns a JSON file but for some reason (probably my mistake) when I use JSON.parse(data);
console.log(data.name) it doesn't work (I know I am doing something wrong here but I don't know how to fix it). Thanks in advance :{)
If you want to get the tracks from the url you have to do this data.tracks.track[0] replace 0 with the needed tracks.
I want to load a tree structure in dojo dynamically with every layer loading(fetching data from server) only after a click to the next layer is made. This would help me to not have the entire tree loaded in memory all together but as someone clicks a level of a tree, only then all the elements of the next level are fetched from the server using ajax requests in dojo.
Can someone help me on how to go about this?
The answer for this is here as follows: using a dijit tree, it has just 3 steps to create a dynamic tree that loads lazily: 1. Create a store, 2. Create a model, 3. Create the tree. Here is the code snippet that I used which worked perfectly:
function createLazyTreeStore()
{
var store = new JsonRest({
target: "location where your file is",
getChildren: function(object) {
// object may just be stub object, so get the full object first and then return it's
// list of children
return this.get(object.id).then(function(fullObject){
console.log(fullObject.children);
return fullObject.children;
});
}
});
return store;
}
function createModel(store)
{
var model = new ObjectStoreModel({
store: store,
getRoot: function(onItem) {
this.store.get("the .php file").then(function(item)
{
onItem(item[0]);
});
},
getChildren: function(object, onComplete, onError)
{
this.store.get("your url to fetch the data with parent child relation").then(function(fullObject) {
object.children = fullObject;
onComplete(object.children);
}, function(error)
{
console.error(error);
onComplete([]);
});
},
mayHaveChildren: function(object) {
return true;
}
});
return model;
}
function createTree(model)
{
var tree = new Tree({
model: model,
style: "your preference",
id: "tree",
});
return tree;
}
Then call these functions in the following order:
var store = createLazyTreeStore();
var model = createModel(store);
var tree = createTree(model);
accordionPane.addChild(tree);