I'm trying to use dynamic insert statements with my database but it fails on character columns. See code below.
library(dplyr)
library(DBI)
library(pool)
library(RSQLite)
df1 <- data.frame(stringsAsFactors = F, id = 1:4, value = letters[1:4])
df2 <- data.frame(stringsAsFactors = F, id = 1:4, value = 100:103)
con <- dbPool(SQLite(), dbname = "test") %>% poolCheckout()
dbWriteTable(con, "with_text", df1, overwrite = T)
dbWriteTable(con, "no_text", df2, overwrite = T)
db1 <- dbReadTable(con, "with_text")
db2 <- dbReadTable(con, "no_text")
new1 <- db1[1,]
new2 <- db2[1,]
query1 <- sprintf(
"INSERT INTO %s (%s) VALUES (%s);",
"with_text",
paste(names(new1), collapse = ", "),
paste(new1, collapse = ", ")
)
query2 <- sprintf(
"INSERT INTO %s (%s) VALUES (%s);",
"no_text",
paste(names(new2), collapse = ", "),
paste(new2, collapse = ", ")
)
db_query1 <- dbSendStatement(con, query1)#fails
dbClearResult(db_query1)
dbReadTable(con, "with_text")
db_query2 <- dbSendStatement(con, query2)
dbClearResult(db_query2)
dbReadTable(con, "no_text")
The #fails line produces this error:
Error in rsqlite_send_query(conn#ptr, statement) : no such column: a
The value of query1 is:
[1] "INSERT INTO with_text (id, value) VALUES (1, a);"
I realize the issue is the lack of single quotes (') around the text value but there has to be a workaround for that. Any help is appreciated. I tried adding column types but couldn't get it to work.
I know that this is a simplified example so I hope that this solution works for your real use case.
`query1 = sprintf("INSERT INTO %s (%s) VALUES (%s,\'%s\');",
"with_text", paste(names(new1), collapse =
","),new1[1,1],new1[1,2])`
I'm editing to add more of an explanation. If you know your second value is going to be text then you can add single quotes into your sprintf to surround that value but you will need to call these values separately instead of calling the entire row and pasting them together. I added \'%s\' escaped single quotes around your second value and then called the values separately.
Related
I have a Shiny app querying large data from a MySQL database but for purposes of this question I will use SQLDF which is similar in syntax on Shiny environment.
A mock up of my app is as below:
library(shiny)
library(dplyr)
library(sqldf)
library(DT)
library(stringr)
df <- data.frame(empName = c("Jon", "Bill", "Maria"),
empID = c("J111", "B222", "M333"),
empAge = c(23, 41, 32),
empSalary = c(21000, 23400, 26800)
)
shinyApp(
ui = fluidPage(
selectizeInput("Search", label = p("Select name"),
choices = as.character(df$empName),
multiple = TRUE),
hr(),
fluidRow(
column(6, DT::dataTableOutput("table1")),
column(6, DT::dataTableOutput("table2"))),
hr(),
hr(),
fluidRow(
column(6, DT::dataTableOutput("table3")),
column(6, DT::dataTableOutput("table4"))
)),
server = function(input, output, session) {
output$table1 = DT::renderDataTable({ df }, options = list(dom = 't'))
df2 <- reactive ({
(df %>% filter(empName %in% input$Search)%>% select(empID))
})
output$table2 = DT::renderDataTable({
req(input$Search)
df2()}, options = list(dom = 't'))
df3 <- reactive({
if (input$Search != "") {
sqldf(paste0("SELECT *
FROM df WHERE empName LIKE '%",input$Search,"%'"))
}})
output$table3 = DT::renderDataTable({
req(input$Search)
df3()}, options = list(dom = 't'))
df4 <- reactive ({
SelectedNames <-stringr::str_c(stringr::str_c("'", input$Search, "'"), collapse = ',')
sqldf(paste0("SELECT empAge, empSalary
FROM df WHERE empName IN (",SelectedNames,") "))
})
output$table4 = DT::renderDataTable({
req(input$Search)
df4()}, options = list(dom = 't'))
})
I am working with MySQL queries. In Table 1 the data displayed is the whole employee dataframe , I cannot do that for thousands of rows from the original data. The Selectize choices are from a dataframe of UNIQUE values of two columns empName and empID
In table 2 I select the employee names from selectizeInput but display the corresponding IDs .
In Table 3, it only shows one ID value corresponding to selected name value from the selectizeInput.
In table 4, the code allows to query other details from multiple selection of the selectizeInput. In this case I pass the multiple selections and use stringr to convert to character strings and pass the variable to the MySQL query.
What I am looking for is to be able to select multiple names from selectizeInput but pass the corresponding multiple employee IDs to the MySQL query to get the results like in Table 4.
Thus basically combine the ability to select names but pass the values of the Id column to allow multiple select in a query.
Any advice or direction towards solving this will be appreciated.
Choose empNames in selectizeInput to get empID in df2() - already done this way in your code. Then use this empID on the original dataframe df to obtain df4 as shown below. Then you get the desired table4. This is what I understood as your expectation.
df4 <- reactive({
df4 <- df %>% subset(empID %in% df2()$empID) %>% select(empID, empAge, empSalary)
})
I want to append new table to original(existing) table without overwriting original table. What is the query code for appending new table?
I have tried the following codes.
## choose one document from vector of strings
file <- x[j]
# read csv for file
doc <- read.csv(file, sep=";")
# indicate number of rows for each doc
n <- nrow(doc)
# create dataframe for doc
df <- data.frame(doc_id = numeric(n), doc_name = character(n), doc_number = character(n), stringsAsFactors = FALSE)
# loop to create df
for(k in 1:nrow(doc)){
df$doc_id[k] <- paste0(df$doc_id[k])
df$doc_name[k] <- paste0(doc$titles[k])
df$doc_number[k] <- paste0(doc$no[k])
}
# query for inserting table to mysql
query1 = sprintf('INSERT IGNORE INTO my_sql_table VALUES ("%s","%s","%s");', df[i,1],df[i,2],df[i,3])
# query for appending table to my_sql_table
query2 = sqlAppendTable(con, "doc", df)
# execute the queries
dbExecute(con, query1)
dbExecute(con, query2)
print(j)
} ## end of for loop
I also tried the following queries for appending, unfortunately, it didn't work.
INSERT IGNORE INTO my_sql_table VALUES ("%s","%s");
INSERT IGNORE INTO tableBackup(SELECT * FROM my_sql_table);
I expect to have appended new_table to original_table without deleting or overwriting the original table.
I have a sql database with one column of ID #'s and a another column that has the corresponding info of each ID. I have a vector that contains the ID #'s that I want the corresponding info to. How do I query only those specific ID's while also getting there corresponding information, and store it into a table?
I've tried for loops, I've tried filtering, and hard coding it.
con <- dbConnect(RSQLite::SQLite(), "data.db")
df<- tbl(con,"kv")
newish <- data.frame(df)
filter(person %in% IDs) %>%
collect()
After connecting call all the ID's in the vector given and extract the corresponding information and store it into a table
If I tired a for loop the table would not print all of the information, but rather only the information of the last ID in the vector, the filtering wouldn't work because it suggested that the vector only had to be of vector length one instead of 90,000. The actual results should be a table that contains only the patient ID #'s and the corresponding information of the people who I have in the vector.
This is a sql problem that you can decompose with R as follows :
# not run
# con <- dbConnect(RSQLite::SQLite(), "data.db")
ids <- paste0("id_", sample(1:100, 10))
id_var <- "id_var" # you can put more ids here with a corresponding data.frame in ids
vars <- paste0("var", 1:10)
db_name <- "mydatabase"
tab_name <- "mytable"
whereq <- paste("where", id_var, "in", paste0("('", paste0(ids, collapse = "', '"), "')") )
rqt <- paste("select", paste(vars, collapse = ", "),
"from", paste0(db_name, ".", tab_name), whereq, ";")
# check the query
rqt
#> [1] "select var1, var2, var3, var4, var5, var6, var7, var8, var9, var10 from mydatabase.mytable where id_var in ('id_99', 'id_1', 'id_72', 'id_86', 'id_65', 'id_59', 'id_67', 'id_4', 'id_82', 'id_2') ;"
# to uncomment
# res <- DBI::dbGetQuery(con, rqt)
# res <- tibble::as_tibble(res) # OR data.table::data.table(res) OR as.data.frame(res)
Created on 2019-06-11 by the reprex package (v0.2.1)
I am new to R and have a database, DatabaseX which contains multiple tables a,b,c,d,etc. I want to use R to count number of rows for a common attribute message_id among all these tables and store it separately.
I can count message_id for all tables using below code:
list<-dbListTables(con)
# print list of tables for testing
print(list)
for (i in 1:length(list)){
query <- paste("SELECT COUNT(message_id) FROM ",list[i], sep = "")
t <- dbGetQuery(con,query)
}
print(t)
This prints :
### COUNT(message_id)
## 1 21519
but I want to keep record of count(message_id) for each individual table. So for example table a = 200, b = 300, c = 500, and etc.
any suggestions how to do this?
As #kaiten65 suggested, one option would be to create a helper function which executes the COUNT query. Outside of your loop I have defined a numeric vector counts which will store the number of records for each table in your database. You can then perform descriptive stats on the tables along with this vector of record counts.
doCountQuery <- function(con, table) {
query <- paste("SELECT COUNT(message_id) FROM ", table, sep = "")
t <- dbGetQuery(con, query)
return(t)
}
list <- dbListTables(con)
counts <- numeric(0) # this will store the counts for all tables
for (i in 1:length(list)) {
count <- doCountQuery(con, list[i])
counts[i] <- count[[1]]
}
You can use functions to achieve what you need. For instance
readDB <- function(db, i){
query <- paste("SELECT COUNT(message_id) FROM ",db, sep = "")
t <- dbGetQuery(con,query)
return(print(paste("Table ", i, " count:", t)
}
list<-dbListTables(con)
for (i in 1:length(list)){
readDB(list[i]);
}
This should print your list recursively but the actual code is in a nice editable function. Your output will be
"Table 1 count: 2519"
"Table 2 count: ---- "
More information on R functions here: http://www.statmethods.net/management/userfunctions.html
i have three variables a,b,c (Actually more than 300 variables in my case)
t<-c(a,b,d)
a<-dbGetQuery(con, "SELECT * FROM a")
b<-dbGetQuery(con, "SELECT * FROM b")
d<-dbGetQuery(con, "SELECT * FROM d")
How can I make a loop to request data from MySQL in R? The existing question does not have the explanation on how to write it into the variable names. I need a,b,c in my environment.
Not tested, but something as below should work.
myTables <- c("a","b","c")
res <- lapply(myTables,
function(myTable){
sqlStatement <- paste("select * from",myTable)
dbGetQuery(con, sqlStatement)
})
names(res) <- myTables