How to join table using a pool connection? - mysql

Question

I am trying to use pool to join two remote tables (City and Country) using below code:
pool <- dbPool(
drv = RMySQL::MySQL(),
dbname = "shinydemo",
host = "shiny-demo.csa7qlmguqrf.us-east-1.rds.amazonaws.com",
username = "guest",
password = "guest"
)
src_pool(pool) %>%
tbl('City') %>%
left_join('Country', by=c('CountryCode'='Code'))
But this is the error I get when run the code:
Error: x and y don't share the same src.
Set copy = TRUE to copy y into x's source (this may be time consuming).
In addition: Warning message:
In force(expr) : You have a leaked pooled object. Destroying it.
Below a working example of the same query using dplyr:
srccon <- src_mysql(
host = "shiny-demo.csa7qlmguqrf.us-east-1.rds.amazonaws.com",
dbname = "shinydemo",
user = "guest",
password = "guest"
)
tbl(srccon, 'City') %>%
left_join(tbl(srccon, 'Country'), by=c('CountryCode'='Code'))
And another example using pool::dbGetQuery
sql <- "SELECT * FROM City LEFT JOIN Country ON (CountryCode=Code)"
dbGetQuery(pool, sql)

Answer 1

I'm not familiar with pool, but the error message:
Error: x and y don't share the same src.
Set copy = TRUE to copy y into x's source (this may be time consuming).
is due to the fact that one of your tables is being manipulated in-database (I'd guess City) and the other is in memory/not in the database ('Country'), but it could be the other way around. In this instance, x and y are your two tables, and the error message is just telling you they're not bothin the same place. By setting copy = TRUE the tibble in y will be (temporarily) copied into wherever x is stored (e.g. either to the database or into RAM) in order to perform the computation.
You could try either bringing both objects into memory using dplyr::collect() which will mean that the join happens in memory; or ensure that both are on your database (in which case the join will occur there instead).
i.e.
# Keep both in RAM
City <- tbl('City') %>% collect()
Country <- tbl('Country') %>% collect()
joined <- City %>% left_join(Country, by = c('CountryCode' = 'Code))
I would guess that the Warning about a leaked pooled object relates to the same issue (one table is in-memory, one is on the database).