Summary
We are processing a document through Google Vision API to generate OCR from the document. However, the Cloud Vision API's text recognition feature is intermittently not recognizing any languages in a document -- even in documents that are 100% a single language.
What have we tried
While the OCR Language Support documentation, recommends not supplying language hints while using DOCUMENT_TEXT_DETECTION in order to gain full, auto-detected language detection, we have tried supplying the language hint of the known language but still no languages are detected.
Additionally, if we take the OCR output that was generating via Cloud Vision and pass that to Google Translate, Cloud Translation, or Amazon Comprehend, they all return the desired language.
Example
Using this document as an example and with the following code (some code removed for brevity/security), Google Cloud Vision does not detect any languages.
const vision = require('#google-cloud/vision').v1;
const client = new vision.ImageAnnotatorClient(authOptions());
const inputConfig = {
mimeType: 'application/pdf',
gcsSource: {
uri: 'our-gcsSource',
},
};
const features = [{ type: 'DOCUMENT_TEXT_DETECTION' }];
const request = {
requests: [
{
inputConfig,
features,
},
],
};
const [operation] = await client.asyncBatchAnnotateFiles(request);
// Additional code removed for brevity
Related
I'm trying to pull current price change % from Google Finance with Google Apps Script. With the following coding, I couldn't figure out why it doesn't pull current price change % (0.72%), though it retrieves "After Hours" price change % (0.081%). Can anyone help me out? Thank you!!
function test() {
var url = 'https://www.google.com/finance/quote/AAPL:NASDAQ';
var res = UrlFetchApp.fetch(url, { muteHttpExceptions: true }).getContentText();
var location = '', sub = '', index = [], price = '', change = [];
// From google finance, scrape whole plain string from first <div class="YMlKec fxKbKc"> tag where current price is.
location = '<div class="YMlKec fxKbKc">';
index = res.indexOf(location);
sub = res.substring(index + location.length, index + location.length + 1000);
// Logger.log(sub);
// Pull current price from the sub variable. This is succesful.
price = sub.substring(0, sub.indexOf('<'));
Logger.log(price);
// Pull current price change, which should be 0.72%. But it didn't work.
// This retrieved only "After Hours" price change %, 0.081%, but not 0.72%, current price change % that I'm looking for.
location = '%';
index = [sub.indexOf(location)];
for (var i = 0; i < index.length; ++i) {
change = sub.substring(index[i] - 5, index[i] + 1);
Logger.log(change)
}
}
Is it possible to use Google Apps Script to get an info from the code I see at DevTools?
First thing that you should be aware is that the code shown in the Elements tab of Chrome Developers Tools is the current DOM but UrlFetchApp.fetch is only able to get the code from the source file referred directly by the URL.
To review the source code you might right click on the webpage and select View page source or go open the file from the Chrome Developers Tool's Sources tab. If the value that you want to read is not there, the best is to use a stock market API, otherwise you have to implement a headless browser.
There have being a several questions about getting data from Google Finance. Most of the question that involves Google Apps Script are related to using the GOOGLEFINANCE built-in function in Google Sheets, where most of the OP didn't reviewed the official help article about GOOGLEFINANCE. From this doc:
Historical data cannot be downloaded or accessed via the Sheets API or Apps Script. If you attempt to do so, you'll see a #N/A error in place of the values in the corresponding cells of your spreadsheet.
The relevance of the above is that Google is taking some measures to prevent that certain data be retrieved automatically.
In order to make Google Apps Script able to "see" what is shown in the web browser using only server-side code you should have to implement a headless browser:
A headless browser is a web browser without a graphical user interface.
Headless browsers provide automated control of a web page in an environment similar to popular web browsers, but they are executed via a command-line interface or using network communication. They are particularly useful for testing web pages as they are able to render and understand HTML the same way a browser would, including styling elements such as page layout, colour, font selection and execution of JavaScript and Ajax which are usually not available when using other testing methods.
Another alternative might be to u use client-side code as the HTML Service could be used to create a dialog/sidebar/web-application to serve it including google.script API support. The later could be used to send to the server-side code some data from the client-side with certain limitations, i.e. Date, Function and DOM objects can't be sent to server side but you might pass them as JSON.
Related
How can I get stock quotes using Google Finance API?
Using Apps Script to scrape javascript rendered web page
Can we use Chrome V8 functionalities from GAS?
You can get current price change using axios and cheerio, the code below shows you how to do this. And check the code in the online IDE
const cheerio = require("cheerio");
const axios = require("axios");
const currencyName = "AAPL:NASDAQ"; // currency name from browser URL
const BASE_URL = "https://www.google.com/finance/quote/";
const AXIOS_OPTIONS = {
headers: {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.157 Safari/537.36",
}, // adding the User-Agent header as one way to prevent the request from being blocked
};
function getCurrentPrice() {
return axios(BASE_URL + currencyName, AXIOS_OPTIONS).then(({ data }) => {
const $ = cheerio.load(data);
const pattern = /data:\[{3}(?<data>\[.{100,500}\])\]{3}, side/gm //https://regex101.com/r/VLKbBt/1
const currencyData = JSON.parse([...data.matchAll(pattern)].map(({groups}) => ({data: groups.data})).find(el=> el.data.includes(currencyName)).data);
const priceChange = currencyData[5][2]
return priceChange;
});
}
getCurrentPrice().then(console.log);
Output:
-0.53461844
And as you can see in the screenshot below, the data that we received is exactly as on the page.
If for some reason you want to make a Google Finance page parser in Python, have a look at the dedicated Google Finance blog post at SerpApi.
Disclaimer, I work for SerpApi
In the older version of javascript I was using managedupload function for uploading big files to s3, which will do the queueing and manage multiparting of files. But in V3 this function is not anywhere in documentation, is that removed? or is there any alternatives? Please help...
In V3 the high level abstractions are moved to functionality specific lib packages while the client packages offer a one to one mapping of the low level public apis.
For S3 the client is in #aws-sdk/client-s3 and the high level operations are in #aws-sdk/lib-storage packages respectively.
Sample upload code for a managed upload would look like the following
const { S3Client } = require("#aws-sdk/client-s3");
const { Upload } = require("#aws-sdk/lib-storage");
const multipartUpload = new Upload({
client: new S3Client({}),
params: {Bucket: 'bucket', Key: 'key', Body: stream},
});
More information here.
I am using the Googel Vision API to detect English.
However, Korean cannot be detected.
I am using bitmap image for recognition. It is source.
TextRecognizer textRecognizer = new TextRecognizer.Builder (this) .build ();
Frame imageFrame = new Frame.Builder ()
.setBitmap (bitmap)
.build ();
SparseArray textBlocks = textRecognizer.detect (imageFrame);
In the case of Google Vision API, if use Language Hint, need to communicate (json) with Google Cloud ???
I have a two-part WPF App and Revit Addin that runs a series of routines to create and activate a new BIM 360 project via the FORGE BIM 360 API, launches Revit and creates and sets up a collection of Revit models from a "seed" Revit model. The primary requirement is to have zero user interaction in this process: to be fully automated.
Given that background, I am having trouble linking the newly saved BIM 360 models to one-another. I have the option to do this either in the Revit Addin with Revit API hooks (preferred) or through the manager WPF App with FORGE API REST calls after the models are created.
Is either one possible?
I have successfully linked with the cached local models as described here and here. However, this does not meet the requirement fully, since when any other user (besides the automation machine user) opens the model the links are Not Found.
Screenshot of links Not Found.
Current "sort-of-working" code:
var wId = GetWorksetId(doc, w);
if (wId != null)
{
string localPath = settings.CloudModels.Where(x => x.ModelName == _linkModelNames[i]).Select(x => x.LocalFilePath).First();
ModelPath path = ModelPathUtils.ConvertUserVisiblePathToModelPath(localPath);
using (var options = new RevitLinkOptions(true))
{
using (var t = new Transaction(doc, w))
{
t.Start();
doc.GetWorksetTable().SetActiveWorksetId(wId);
using (var result = RevitLinkType.Create(doc, path, options))
{
_ = RevitLinkInstance.Create(doc, result.ElementId);
}
t.Commit();
linkPlaced++;
}
}
}
I was able to get the correct ModelPath via the Revit API thanks to this helpful tip. I can save this value in my Addin, close the model, and access the property later after opening a different model (saved in the CloudModels class referenced in the linq statement in the code snippet above). Unfortunately for me RevitLinkType.Create() that takes a ModelPath does not accept the cloud path, so I may have hit another dead end. Unless it is possible with an ExternalResourceReference. Has anyone tried this option? If so, how do you assemble a Revit ExternalResourceReference? I am not familiar with this process, and looking over this course from AU 2017, I don't see that it necessarily applies to BIM 360 cloud models. A BIM 360 cloud example would be very helpful if this is possible.
Alternate strategy: I do not see any reference to loading links in the FORGE Data Management API or other FORGE APIs. If I have somehow missed it, please share a link.
Any help would be very much appreciated!!
EDIT: I have since found these two (1) (2) similar questions that, at least for my purposes, were not answered satisfactorily. Any updates I should be aware of?
As of now (Jan 2020), unfortunately, we do not have a Link API for cloud models. It is on the roadmap.
Revit API 2022 docs mention that
The methods:
RevitLinkType.Create(Document, ModelPath, RevitLinkOptions)
RevitLinkType.LoadFrom(ModelPath, WorksetConfiguration)
have been enhanced to support creation of new cloud model Revit links.
You may use ModelPathUtils.ConvertCloudGUIDsToCloudPath() to
create a cloud path to use as an argument to these methods.
Additional Resource:
This Youtube Video showcases an example and its respective Github repo.
For Revit 2021 and below, you can use ExternalResourceReference() as a workaround, but I've noticed that this is not always reliable.
Its also mentioned in the documentation of InSessionPath property to not rely on this property:
Do not rely on this path (InSessionPath ) to look up an ExternalResourceReference, as the path is neither unique nor stable. It isn't unique because multiple servers might use the same server name and display name format. It isn't stable because some servers allow renaming, and because a server might change its name at some point.
Below is the code to do that:
var linkCloudPath = doc.GetCloudModelPath(); // the cloudpath of a BIM360 model
Guid linkedmodelguid = linkCloudPath.GetModelGUID();
Guid linkedprojectguid = linkCloudPath.GetProjectGUID();
Dictionary<string, string> Dictionary_ExternalResource = new Dictionary<string, string>(){
{"LinkedModelModelId", modelGuid.ToString()},
{"LinkedModelProjectId", projGuid.ToString()}
};
Dictionary<string, Guid> servers = new Dictionary<string, Guid>();
foreach (var service in ExternalServiceRegistry.GetServices())
{
if (service.Name == "External Resource Service")
{
IList<Guid> server_ids = service.GetRegisteredServerIds();
foreach (var server_id in server_ids)
{
servers.Add(service.GetServer(server_id).GetName(), server_id);
}
}
}
Guid BIM360ServerID = servers["BIM 360"];
ExternalResourceReference ERS = new ExternalResourceReference(BIM360ServerID, Dictionary_ExternalResource, "", "");
RevitLinkOptions options = new RevitLinkOptions(false);
LinkLoadResult result = RevitLinkType.Create(gcdoc, ERS, options);
RevitLinkInstance.Create(gcdoc, result.ElementId);
Please note that this seems to be working fine in Revit 2020, but not in Revit 2021.
I believe it is possible to create links to the Cloud Models in Revit 2019 or higher (with ModelPathUtils.ConvertCloudGUIDsToCloudPath()). You'll need the ProjectGUID and ModelGUID to make the cloud model path.
Regarding the ExternalResource approach, that also works - but it's super messy - you can read the properties associated with existing BIM360 links and you'll see how an ExternalResource is defined for BIM360 links.
Finally - as of today, the Forge Design Automation for Revit approach would not work for you at all:
1. Not possible to open a live cloud workshared model (only published/uploaded models).
2. No network access while you're running in a Design Automation for Revit session.
Good luck...
-Matt
How can I create a javascript function to put in Cloud Code of parse.com, which downloads the content of the class in a json file?
I've read through the Cloud Code Documentation at parse.com https://www.parse.com/docs/cloud_code_guide but I haven't found any examples similar to what I need to do.
You can use this for example (there a few ways). Remove // to limit the number of objects (2 or whatever). "english" being my class.
Then you need to read this function from your platform (mobile desktop etc) which you also need to know how to do. What platform do you use? I only know Corona SDK.
Parse.Cloud.define("getall", function(request, response) {
var query = new Parse.Query("english");
//query.limit(2)
query.find({
success: function(results) {
response.success(results);
},