I have my Report model:
class Report < ActiveRecord::Base
belongs_to :user
attr_accessible :ready_status, :document
mount_uploader :document, DocumentUploader
def attach( report_file )
self.update_attributes( :document => File.open( report_file ), :ready_status => true )
end
end
This model has attach metod, which i use to save document and other param. Now i want to test that this function works.
/spec/models/report_spec.rb
# encoding: utf-8
require 'spec_helper'
describe Report do
before(:each) do
#user = User.make!
end
...
context "File's saving" do
before(:each) do
#report = #user.reports.create
#csv_report_file = "#{Rails.root}/spec/files/report.csv"
end
it "CSV should be saved" do
csv_report_filename = #csv_report_file.split("/").last
#report.attach #csv_report_file
#report.reload
#report.document.file.filename.should == csv_report_filename
end
end
end
When i try to saving file from /spec/files i get such error:
Report File's saving CSV should be saved
Failure/Error: #report.document.file.filename.should == csv_report_filename
NoMethodError:
undefined method `filename' for nil:NilClass
But when i try another file from another folder (for example "#{Rails.root}/samples/my-report.csv") then my test passes.
How can i fix that?
Oh, i found the answer. Carrierwave doesn't save empty file and i had one. When i added some data in the file (/spec/files/report.csv) my problem has gone.
Related
I have a controller that render a json with data from models. When i enter route to get it it do nothing and just show and error in console.
Controller:
class Api::ForecastController < ApplicationController
before_action :set_hourly_forecast, only: %i[ show edit update destroy ]
def index
respond_to do |format|
format.html do
#hourly_forecasts = HourlyForecast.where(forecast_location_id: params[:forecast_location_id]).paginate(:page => params[:page], :per_page=>24) if params[:forecast_location_id].present?
end
format.json do
weather_service = WeatherService.where(name: params[:name])
#forecast_location = ForecastLocation.where(weather_service_id: weather_service)#& municipality: #municipality.name)
#hourly_forecasts = HourlyForecast.where(forecast_location_id: forecast_location.id ).paginate(:page => params[:page], :per_page=>24) if params[:forecast_location_id].present?
end
end
end
private
# Use callbacks to share common setup or constraints between actions.
def set_hourly_forecast
#hourly_forecast = HourlyForecast.find(params[:id])
end
# Only allow a list of trusted parameters through.
def hourly_forecast_params
params.require(:hourly_forecast).permit(:forecast_location_id, :date, :temperature, :humidity, :rain, :rain_probability, :wind_speed, :wind_direction)
end
end
Error:
> Started GET "/api/forecast.json?name=dark_sky" for 127.0.0.1 at 2022-04-20 18:33:29 +0200
Processing by Api::ForecastController#index as JSON
Parameters: {"name"=>"dark_sky"}
No template found for Api::ForecastController#index, rendering head :no_content
Completed 204 No Content in 53ms (ActiveRecord: 6.2ms | Allocations: 4983)
The route i use its
127.0.0.1:3000/api/forecast.json?name=dark_sky
So the output should be a json with all hourly model.
But it just do nothing and on console it does get and the select but jumps this error of template, i dont understand it since im new on rails.
If need more controllers, models or anything ask in comments.
You have to have a separate template file to render json index.json.jbuilder for example.
# app/views/api/forecasts/index.json.jbuilder
json.array! #hourly_forecasts do |hourly_forecast|
json.extract! hourly_forecast, :id, :forecast_location_id, :date, :temperature, :humidity, :rain, :rain_probability, :wind_speed, :wind_direction
json.url api_forecast_url(hourly_forecast, format: :json)
end
https://github.com/rails/jbuilder
If you don't need to customize rendered json too much, render json inline in the controller
format.json do
weather_service = WeatherService.where(name: params[:name])
#forecast_location = ForecastLocation.where(weather_service_id: weather_service)#& municipality: #municipality.name)
#hourly_forecasts = HourlyForecast.where(forecast_location_id: forecast_location.id ).paginate(:page => params[:page], :per_page=>24) if params[:forecast_location_id].present?
render json: #hourly_forecasts
end
https://guides.rubyonrails.org/layouts_and_rendering.html#rendering-json
https://api.rubyonrails.org/classes/ActiveModel/Serializers/JSON.html
I am using gem roo to import CSV data. It works smoothly, until the point where there is an association, and am hoping that roo can translate the string into the corresponding integer value in the association. In my case, I have a Staff model which belongs to State.
class State < ApplicationRecord
has_many :staffs
end
class Staff < ApplicationRecord
belongs_to :state
end
This means that I have state_id column in the staffs table. In my CSV, however, the end user has the names of the states, which correspond to the ones in the states tables. When I try to import the CSV, I get the error:
ActiveRecord::AssociationTypeMismatch in StaffsImportsController#create
State(#134576500) expected, got "Texas" which is an instance of String(#20512180)
The highlighted source is:
staff.attributes = row.to_hash
Is it possible for gem roo to translate 'Texas' in the csv file to, say, id 2, instead of the end user doing a lot of translation work before uploading the data?
Here is staffs_imports.rb
class StaffsImport
include ActiveModel::Model
require 'roo'
attr_accessor :file
def initialize(attributes={})
attributes.each { |name, value| send("#{name}=", value) }
end
def persisted?
false
end
def open_spreadsheet
case File.extname(file.original_filename)
when ".csv" then Csv.new(file.path, nil, :ignore)
when ".xls" then Roo::Excel.new(file.path, nil, :ignore)
when ".xlsx" then Roo::Excelx.new(file.path)
else raise "Unknown file type: #{file.original_filename}"
end
end
def load_imported_staffs
spreadsheet = open_spreadsheet
header = spreadsheet.row(1)
(2..spreadsheet.last_row).map do |i|
row = Hash[[header, spreadsheet.row(i)].transpose]
staff = Staff.find_by_national_id(row["national_id"]) || Staff.new
staff.attributes = row.to_hash
staff
end
end
def imported_staffs
#imported_staffs ||= load_imported_staffs
end
def save
if imported_staffs.map(&:valid?).all?
imported_staffs.each(&:save!)
true
else
imported_staffs.each_with_index do |staff, index|
staff.errors.full_messages.each do |msg|
errors.add :base, "Row #{index + 6}: #{msg}"
end
end
false
end
end
end
And finally the staff_imports_controller.rb:
class StaffsImportsController < ApplicationController
def new
#staffs_import = StaffsImport.new
end
def create
#staffs_import = StaffsImport.new(params[:staffs_import])
if #staffs_import.save
flash[:success] = "You have successfully uploaded your staff!"
redirect_to staffs_path
else
render :new
end
end
end
Any help/clues will be highly appreciated.
I managed to get a solution to this, thanks to a wonderfully detailed question and great answer provided here Importing CSV data into Rails app, using something other then the association "id"
I've written a simple plugin that generates a small JSON file
module Jekyll
require 'pathname'
require 'json'
class SearchFileGenerator < Generator
safe true
def generate(site)
output = [{"title" => "Test"}]
path = Pathname.new(site.dest) + "search.json"
FileUtils.mkdir_p(File.dirname(path))
File.open(path, 'w') do |f|
f.write("---\nlayout: null\n---\n")
f.write(output.to_json)
end
# 1/0
end
end
end
But the generated JSON file gets deleted every time Jekyll runs to completion. If I uncomment the division by zero line and cause it to error out, I can see that the search.json file is being generated, but it's getting subsequently deleted. How do I prevent this?
I found the following issue, which suggested adding the file to keep_files: https://github.com/jekyll/jekyll/issues/5162 which worked:
The new code seems to avoid search.json from getting deleted:
module Jekyll
require 'pathname'
require 'json'
class SearchFileGenerator < Generator
safe true
def generate(site)
output = [{"title" => "Test"}]
path = Pathname.new(site.dest) + "search.json"
FileUtils.mkdir_p(File.dirname(path))
File.open(path, 'w') do |f|
f.write("---\nlayout: null\n---\n")
f.write(output.to_json)
end
site.keep_files << "search.json"
end
end
end
Add your new page to site.pages :
module Jekyll
class SearchFileGenerator < Generator
def generate(site)
#site = site
search = PageWithoutAFile.new(#site, site.source, "/", "search.json")
search.data["layout"] = nil
search.content = [{"title" => "Test 32"}].to_json
#site.pages << search
end
end
end
Inspired by jekyll-feed code.
I have the code that implement csv upload like this:
def Hotel.import(file)
CSV.foreach(file.path, headers: true) do |row|
product = find_by_id(row["id"]) || new
product.attributes = row.to_hash
product.save
end
end
def import
Hotel.import(params[:file])
redirect_to root_url, notice: "Product was successfully Imported."
end
so how do I write rspec test for this?
There are lots of ways to write controller specs. There are many good resources online outlining how to write them in different styles. I suggest starting with the RSpec docs on controller specs:
https://github.com/rspec/rspec-rails#controller-specs
https://www.relishapp.com/rspec/rspec-rails/v/2-14/docs/controller-specs
In general they go something like:
require "spec_helper"
describe ProductsController do
describe "POST #import" do
it "redirects to the home page" do
allow(Hotel).to receive(:import).with("foo.txt")
post :import, file: "foo.txt"
expect(response).to redirect_to root_url
end
it "adds a flash notice" do
allow(Hotel).to receive(:import).with("foo.txt")
post :import, file: "foo.txt"
expect(flash[:notice]).to eq "Product was successfully imported."
end
it "imports the hotel file" do
expect(Hotel).to receive(:import).with("foo.txt")
post :import, file: "foo.txt"
end
end
end
If any one needed model tests for rspec.
require 'rails_helper'
RSpec.describe Product, type: :model do
describe 'import' do
before :each do
#file = fixture_file_upload('data.csv', 'csv')
end
context 'when file is provided' do
it 'imports products' do
Product.import(#file)
expect(Product.find_by(part_number: '0121G00047P').description)
.to eq 'GALV x FAB x .026 x 29.88 x 17.56'
end
end
end
end
i want to upload CSV files through the activeadmin panel.
on the index page from the resource "product" i want a button next to the "new product" button with "import csv file".
i dont know where to start.
in the documentation is something about collection_action, but with the code below i have no link at the top.
ActiveAdmin.register Post do
collection_action :import_csv, :method => :post do
# Do some CSV importing work here...
redirect_to :action => :index, :notice => "CSV imported successfully!"
end
end
anyone here who use activeadmin and can import csv data?
Continuing from Thomas Watsons great start to the answer which helped me get my bearings before figuring the rest of it out.
The code blow allows not just CSV upload for the example Posts model but for any subsequent models thereafter. all you need to do is copy the action_item ands both collection_actions from the example into any other ActiveAdmin.register block and the functionality will be the same. hope this helps.
app/admin/posts.rb
ActiveAdmin.register Post do
action_item :only => :index do
link_to 'Upload CSV', :action => 'upload_csv'
end
collection_action :upload_csv do
render "admin/csv/upload_csv"
end
collection_action :import_csv, :method => :post do
CsvDb.convert_save("post", params[:dump][:file])
redirect_to :action => :index, :notice => "CSV imported successfully!"
end
end
app/models/csv_db.rb
require 'csv'
class CsvDb
class << self
def convert_save(model_name, csv_data)
csv_file = csv_data.read
CSV.parse(csv_file) do |row|
target_model = model_name.classify.constantize
new_object = target_model.new
column_iterator = -1
target_model.column_names.each do |key|
column_iterator += 1
unless key == "ID"
value = row[column_iterator]
new_object.send "#{key}=", value
end
end
new_object.save
end
end
end
end
note: this example does a check to see whether or not the first column is an ID column, it then skips that column as rails will assign an ID to the new object (see example CSV below for reference)
app/views/admin/csv/upload_csv.html.haml
= form_for :dump, :url=>{:action=>"import_csv"}, :html => { :multipart => true } do |f|
%table
%tr
%td
%label{:for => "dump_file"}
Select a CSV File :
%td
= f.file_field :file
%tr
%td
= submit_tag 'Submit'
app/public/example.csv
"1","TITLE EXAMPLE","MESSAGE EXAMPLE","POSTED AT DATETIME"
"2","TITLE EXAMPLE","MESSAGE EXAMPLE","POSTED AT DATETIME"
"3","TITLE EXAMPLE","MESSAGE EXAMPLE","POSTED AT DATETIME"
"4","TITLE EXAMPLE","MESSAGE EXAMPLE","POSTED AT DATETIME"
"5","TITLE EXAMPLE","MESSAGE EXAMPLE","POSTED AT DATETIME"
note: quotations not always needed
Adding a collection_action does not automatically add a button linking to that action. To add a button at the top of the index screen you need to add the following code to your ActiveAdmin.register block:
action_item :only => :index do
link_to 'Upload CSV', :action => 'upload_csv'
end
But before calling the collection action you posted in your question, you first need the user to specify which file to upload. I would personally do this on another screen (i.e. creating two collection actions - one being a :get action, the other being your :post action). So the complete AA controller would look something like this:
ActiveAdmin.register Post do
action_item :only => :index do
link_to 'Upload posts', :action => 'upload_csv'
end
collection_action :upload_csv do
# The method defaults to :get
# By default Active Admin will look for a view file with the same
# name as the action, so you need to create your view at
# app/views/admin/posts/upload_csv.html.haml (or .erb if that's your weapon)
end
collection_action :import_csv, :method => :post do
# Do some CSV importing work here...
redirect_to :action => :index, :notice => "CSV imported successfully!"
end
end
#krhorst, I was trying to use your code, but unfortunately it sucks on big imports. It eat so much memory =( So I decided to use own solution based on activerecord-import gem
Here it is https://github.com/Fivell/active_admin_import
Features
Encoding handling
Support importing with ZIP file
Two step importing (see example2)
CSV options
Ability to prepend CSV headers automatically
Bulk import (activerecord-import)
Ability to customize template
Callbacks support
Support import from zip file
....
Based on ben.m's excellent answer above I replaced the csv_db.rb section suggested with this:
require 'csv'
class CsvDb
class << self
def convert_save(model_name, csv_data)
begin
target_model = model_name.classify.constantize
CSV.foreach(csv_data.path, :headers => true) do |row|
target_model.create(row.to_hash)
end
rescue Exception => e
Rails.logger.error e.message
Rails.logger.error e.backtrace.join("\n")
end
end
end
end
While not a complete answer I did not want my changes to pollute ben.m's answer in case I did something egregiously wrong.
expanding on ben.m's response which I found very useful.
I had issues with the CSV import logic (attributes not lining up and column iterator not functioning as required) and implemented a change which instead utilizes a per line loop and the model.create method. This allows you to import a .csv with the header line matching the attributes.
app/models/csv_db.rb
require 'csv'
class CsvDb
class << self
def convert_save(model_name, csv_data)
csv_file = csv_data.read
lines = CSV.parse(csv_file)
header = lines.shift
lines.each do |line|
attributes = Hash[header.zip line]
target_model = model_name.classify.constantize
target_model.create(attributes)
end
end
end
end
So your imported CSV file can look like this (use to match up with model attributes):
importExample.csv
first_name,last_name,attribute1,attribute2
john,citizen,value1,value2
For large excel which takes time on normal process, I created a gem that process Excel sheets using an active job and display results using action cable(websockets)
https://github.com/shivgarg5676/active_admin_excel_upload
Some of the solutions above worked pretty well. I ran into challenges in practice that I solved here below. The solved problems are:
Importing CSV data with columns in different orders
Preventing errors caused by hidden characters in Excel CSVs
Resetting the database primary_key so that the application can continue to add records after the import
Note: I took out the ID filter so I could change IDs for what I'm working on, but most use cases probably want to keep it in.
require 'csv'
class CsvDb
class << self
def convert_save(model_name, csv_data)
csv_file = csv_data.read
csv_file.to_s.force_encoding("UTF-8")
csv_file.sub!("\xEF\xBB\xBF", '')
target_model = model_name.classify.constantize
headers = csv_file.split("\n")[0].split(",")
CSV.parse(csv_file, headers: true) do |row|
new_object = target_model.new
column_iterator = -1
headers.each do |key|
column_iterator += 1
value = row[column_iterator]
new_object.send "#{key.chomp}=", value
end
new_object.save
end
ActiveRecord::Base.connection.reset_pk_sequence!(model_name.pluralize)
end
end
end