It seems that SQL Alchemy can connect to MySQL table running on Google Cloud SQL. However, I spent time to look for wrapper of Google Cloud Bigtable, a NoSQL database, and could not find anything enough mature.
Just wondering how to manage Google Cloud Bigtable from SQL Alchemy.
There is some Python API to connect to Big Table Cloud:
https://googlecloudplatform.github.io/google-cloud-python/stable/
The google-cloud library is pip install-able:
$ pip install google-cloud
Cloud Datastore
from google.cloud import datastore
client = datastore.Client()
key = client.key('Person')
entity = datastore.Entity(key=key)
entity['name'] = 'Your name'
entity['age'] = 25
client.put(entity)
However, this is still not integrated through SQL Alchemy, this is not clear that Schema can be easily integrated.
This is not possible, because SQLAlchemy can only manage SQL-based RDBMS-type systems, while Bigtable (and HBase) are NoSQL, non-relational systems.
Here's my detailed response on a feature request that was filed for the Google Cloud Python library project which has more context and alternative suggestions:
The integration between SQLAlchemy and Google Cloud Bigtable would
have to be done in SQLAlchemy. I was going to file a bug on SQLAlchemy
on your behalf, but looks like you've already filed a feature
request
and it was closed as wontfix:
unfortunately Google bigtable is non-relational and non-SQL, SQLAlchemy does not have support for key/value stores.
and a previous email thread on the sqlalchemy#
list
about adding support for NoSQL databases like HBase (which is very
similar to Bigtable) ended up without any answers.
Thus, I am afraid we won't be able to help you use SQLAlchemy together
with Bigtable.
That said, as an alternative, consider using Apache
Hue, which works with Apache HBase, and can be
made to work similarly with Bigtable. We don't have a simple howto for
how to connect Apache Hue to Cloud Bigtable yet, but I imagine it can
be done as follows:
Apache Hue -> (a: Thrift API) -> Apache HBase Thrift proxy -> (b: gRPC API) -> Google Cloud Bigtable
The first connection (a) should work out-of-the-box for Hue and
HBase. The second connection can use the Google Cloud Bigtable Java
client for
HBase.
This is not as complicated as it looks, although there are several
parts to connect together to make it all work.
Apache Hue -> (gRPC API) -> Google Cloud Bigtable
This could be done using the Google Cloud Bigtable Java client for
HBase, but
it requires Apache Hue to use the HBase 1.x API (which I believe is
not yet the case, I believe it's using 0.9x API and/or Thrift), so I
would recommend following option (1) above for now instead.
Hope this is helpful.
Related
I will create a mobile app for Android and iOS. The purpose of the app is to access just a segment of the MySQL database, that part is for customers (the rest of the database is used by employees via my web application).
Is there a way the for FlutterFlow app to access MySQL database instead of FireBase Storage? If not, what are the alternatives?
Though flutterflow has default integration with Firebase firestore database,
below 2 approaches can be used.
API integration approach can be used to use MYSQL database.
Please refer below flow:
You can have backend Server which will host the API.
This backend server will have integration with MYSQL DB
API's will be consumed in flutterflow app
https://docs.flutterflow.io/data-and-backend/api-calls
Custom code approach (Though have not used it before)
https://docs.flutterflow.io/customizing-your-app/manage-custom-code-in-github
According to this
There is a Go library for Cloud SQL.
Google Cloud SQL on App Engine:
user#cloudsql(project-id:instance-name)/dbname
But according the GAE site, you can (maybe should?) use only java or python to connect to Cloud SQL: https://developers.google.com/cloud-sql/faq#languages
Can I use languages other than Java or Python?
Only Java and Python are supported for Google Cloud SQL.
I'm determining if GAE is a good fit for my Go application. Is Google discouraging the use of 3rd party libraries to connect to Cloud SQL?
Not being able to use Cloud SQL would be a deal breaker for me and will have me looking for another platform to host my Go application.
Using Google Cloud SQL with App Engine Go SDK
This document describes how to use Google Cloud SQL instances with the
App Engine Go SDK.
Cloud SQL with Go-MySQL-Driver
We would like to add support for Cloud SQL to
github.com/go-sql-driver/mysql.
There shouldn't be much to do. The appengine/cloudsql package 1
provides a Dial function that takes a CloudSQL instance name and
returns a net.Conn; you just have to arrange for your mysql driver to
call that (or have some way of taking a net.Conn).
You could follow what we did for ziutek's mymysql package 2.
Why do you think you have will have a problem?
I belive you can use 3rd packages, here is the document in Go specific App Engine API:
https://developers.google.com/appengine/docs/go/cloud-sql/reference
Applications should use this package through database/sql: Select a
pure Go MySQL driver that supports this package, and use sql.Open with
protocol "cloudsql" and an address of the CloudSQL instance. The exact
format of the second argument to sql.Open is driver-dependent; consult
the driver's documentation for details.
import "database/sql"
import _ "<some mysql package>"
So I think "" can be https://github.com/go-sql-driver/mysql
I am developing a GWT project that is hosted in GAE, and I would like to use an external MySQL database for persistence, instead of Datastore. Is it possible to do this? Is it compatible with RequestFactory?
Thanks
GAE offers the ability to use Google Cloud SQL instead of the High Replication Datastore.
RequestFactory does not make assumptions about the datastore you are using so there should not be any problem using it with Google Cloud SQL. You can also use both at the same time.
You cannot connect to a database server hosted anywhere else, unless you would create an interface that works on the HTTP level... which sounds like a really bad idea for several reasons.
We would like to use Google App Engine for a Django project for rapid development and scalable deployment. We'd also like to use an existing MySQL server as a database. In theory this would eliminate the problems with using the Google App Engine storage solutions and Django.
Unfortunately, although the app works locally, I get the following error when I deploy:
ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb
From posts by people wanting to do this with Java I understand that I can't open a MySQL port from GAE. This being the case, is there a drop-in solution for accessing a remote MySQL database (using a technique like tunnelling over HTTP) in Python that would let me keep using the Django ORM features?
You cannot access your own Mysql, however you can use Google cloud Sql.
I just wanted to ask if we can access an external MySQL server from Google App Engine...
The only way to communicate with other hosts is by using UrlFetch that only provides HTTP and HTTPS requests.
So, you can't do it out of the box.
Anyway, if you really need to access an external MySQL server database, you should consider to expose it through a Web API (RESTful, Soap web-services for example).
In this way your data would be available also via UrlFetch.
I am still in the learning phase of all this, but I am fairly certain you can do this now a few ways:
Link Apps Scripts to App Engine and use the JDBC
Link it to Google
Cloud SQL Store your SQL database on Google Cloud Store
Connect Apps Scripts via spreadsheet scripting
"Google Apps Script has the ability to make connections to databases via JDBC with the Jdbc Service. The current support extends to MySQL, Microsoft SQL Server and Oracle. Apps Script makes it easy to connect to databases hosted on Google Cloud SQL, but also works with other cloud hosting platforms and even local databases." https://developers.google.com/apps-script/jdbc
(edited for structure)
It is still not possible to native connect GAE to an external MySQL server the only exception is Googles Cloud SQL. We are using it in our production environment and like the experience. Stable and the performance is good.
GAEs own database scales well and we are using it in most situations, but in cases where we need to ask more complex questions or need aggregate functions, we use Googles Cloud SQl.
These answers are a bit outdated. Google App Engine instances can connect to external database servers.
The ability to connect externally requires that the account the App Engine is running under be a "paid account" a/k/a "billing enabled".
References:
-https://cloud.google.com/appengine/docs/php/using-third-party-databases
-https://cloud.google.com/appengine/docs/php/runtime#PHP_Functions_that_requires_billing_enabled