Web page linking to SQL2000 - taking too long to fully load - html

I have an internal web page that connects to an SQL2000 database, pulls the data and then displays it on a table. The SQL query takes on average 3 seconds to get the full results (in this instance 59 rows).
select Call_Ref, per_data7, dbo.dateonly(Scheduled_Date_Time) as sched_date, Call_Status_Description, Add1, Add2, Post_Code, contract_short_name, Call_Type_Description, LUCFC_Description, sched_colour_code
from Calls with (nolock)
inner join Clients with (nolock) on Link_to_Client=Client_Ref
left join Personnel with (nolock) on Last_Allocated_To=Pers_Ref
left join LU_Call_Types with (nolock) on Call_Type=Call_Type_Code
left join Personnel_More with (nolock) on Last_Allocated_To=PER_Link_to_Pers_Ref
left join LU_Call_Fault_codes with (nolock) on call_fault_code_1=LUCFC_Code
left join LU_Call_Status with (nolock) on Last_Event_Status=Call_Status_Code
left join call_more on call_ref=callm_link_to_call
left join contractids on link_to_contract_header=contract_ref
where dbo.dateonly(Scheduled_Date_Time) between '09/09/2013' and '13/09/2013'
and Call_Type in ('BC','IN')
and PER_Data7 in ('Team 1','Team 2','Team 3','Team 4','Team 5','Team 6','Team 7','Team 8','Team 9','Team 10','Gas1')
and Call_Status_Description in ('Allocated','Reported Done','Complete')
or CALLM_Data21 between '19/08/2013' and '23/08/2013'
and CALLM_Data22 in ('Team 1','Team 2','Team 3','Team 4','Team 5','Team 6','Team 7','Team 8','Team 9','Team 10','Gas1')
and link_to_contract_header = 'BGAS-1'
and call_type not in ('BC','IN')
and call_status_description not in ('Cancelled')
order by per_data7, dbo.dateonly(Scheduled_Date_Time)
I am then displaying those results in a table, using do/while loops and if statements to filter the results into where they should be displayed in the table - the problem is that each of those takes an additional 3 seconds to display - thus the complete web page takes around 3 and half minutes to fully load (i had to extend the timeout to allow to fully load).
I need to get this down to a usable amount of time.
The webserver is Windows IIS and the web page is basic html, classic-asp with some VB.
It server also has PHP installed, but i am not sure which version.
I have attached the files in a ZIP for anyone to look at.
https://www.dropbox.com/s/qbfp0cswo403e4u/files.zip
So, guys.... what can I do to get this page to load in a timescale that makes it actually usable?
A viewable version of the page is here http://195.171.121.111/week0909-test.asp which I will leave there for a day or so.

The user defined function function in the where and order by clause is probably causing the slow-down for SQL.
You should be able to pull the query data once, and then build up in memory hashes of the Teams and Dates. Another hash should contain lists of the entries for a cross hash between the team and dates, then you can just iterate over the data set once instead of (team-names.count * dates.count) times.

Related

Access Report from Query inexplicably multiplies columns

I have a weird problem. First off, I'm not well versed in Access, but I'm using it for a project to track visibility of log sources and detections and map them to the MITRE ATT&CK Framework. I tables such as Techniques to track the MITRE Techniques, Data_Sources to track the various sources used by those Techniques, and Data_Sources_Coverage that has a visibility ranking for each data source for the company. I have a query built with the below SQL (probably a little messy):
SELECT Technique.Name, Technique.ID, Technique.[Tactic ID], Count(Technique.[Data Sources].Value) AS Technique_Sources, Count(Data_Source_Coverage.[Data Source]) AS Sources_Visible
FROM (Data_Sources INNER JOIN Technique ON Data_Sources.Name = Technique.[Data Sources].Value) INNER JOIN Data_Source_Coverage ON Data_Sources.Name = Data_Source_Coverage.[Data Source]
WHERE (Data_Source_Coverage.Coverage)>0)
GROUP BY Technique.Name, Technique.ID, Technique.[Tactic ID];
The query works great, prints the results as I expect them. However, when I generate a report from that query things go off the rails. The report inexplicably (I did nothing to the data), multiplies the Technique_Sources column by the Sources_Visible column and displays that in the Technique_Sources Column, and then every entry in the Sources_Visible column becomes a '1'. Pics below. Thanks for the help.

How can this query be optimized for speed?

This query creates an export for UPS from the deliveries history:
select 'key'
, ACC.Name
, CON.FullName
, CON.Phone
, ADR.AddressLine1
, ADR.AddressLine2
, ADR.AddressLine3
, ACC.Postcode
, ADR.City
, ADR.Country
, ACC.Code
, DEL.DeliveryNumber
, CON.Email
, case
when CON.Email is not null
then 'Y'
else 'N'
end
Ship_Not_Option
, 'Y' Ship_Not
, 'ABCDEFG' Description_Goods
, '1' numberofpkgs
, 'PP' billing
, 'CP' pkgstype
, 'ST' service
, '1' weight
, null Shippernr
from ExactOnlineREST..GoodsDeliveries del
join ExactOnlineREST..Accounts acc
on ACC.ID = del.DeliveryAccount
join ExactOnlineREST..Addresses ADR
on ADR.ID = DEL.DeliveryAddress
join ExactOnlineREST..Contacts CON
on CON.ID = DEL.DeliveryContact
where DeliveryDate between $P{P_SHIPDATE_FROM} and $P{P_SHIPDATE_TO}
order
by DEL.DeliveryNumber
It takes many minutes to run. The number of deliveries and accounts grows with several hundreds each day. Addresses and contacts are mostly 1:1 with accounts. How can this query be optimized for speed in Invantive Control for Excel?
Probably this query is run at most once every day, since the deliverydate does not contain time. Therefore, the number of rows selected from ExactOnlineREST..GoodsDeliveries is several hundreds. Based upon the statistics given, the number of accounts, deliveryaddresses and contacts is also approximately several hundreds.
Normally, such a query would be optimized by a solution such as Exact Online query with joins runs more than 15 minutes, but that solution will not work here: the third value of a join_set(soe, orderid, 100) is the maximum number of rows on the left-hand side to be used with index joins. At this moment, the maximum number on the left-hand side is something like 125, based upon constraints on the URL length for OData requests to Exact Online. Please remember the actual OData query is a GET using an URL, not a POST with unlimited size for the filter.
The alternatives are:
Split volume
Data Cache
Data Replicator
Have SQL engine or Exact Online adapted :-)
Split Volume
In a separate query select the eligible GoodsDeliveries and put them in an in-memory or database table using for instance:
create or replace table gdy#inmemorystorage as select ... from ...
Then create a temporary table per 100 or similar rows such as:
create or replace table gdysubpartition1#inmemorystorage as select ... from ... where rowidx$ between 0 and 99
... etc for 100, 200, 300, 400, 500
And then run the query several times, each time with a different gdysubpartition1..gdysubpartition5 instead of the original from ExactOnlineREST..GoodsDeliveries.
Of course, you can also avoid the use of intermediate tables by using an inline view like:
from (select * from goodsdeliveries where date... limit 100)
or alike.
Data Cache
When you run the query multiple times per day (unlikely, but I don't know), you might want to cache the Accounts in a relational database and update it every day.
You can also use a 'local memorize results clipboard andlocal save results clipboard to to save the last results to a file manually and later restore them usinglocal load results clipboard from ...andlocal insert results clipboard in table . And maybe theninsert into from exactonlinerest..accounts where datecreated > trunc(sysdate)`.
Data Replicator
With Data Replicator enabled, you can have replicas created and maintained automatically within an on-premise or cloud relational database for Exact Online API entities. For low latency, you will need to enable the Exact webhooks.
Have SQL Engine or Exact adapted
You can also register a request to have the SQL engine to allow higher number in the join_set hint, which would require addressing the EOL APIs in another way. Or register a request at Exact to also allow POST requests to the API with the filter in the body.

Access 'No Records Found' when Records Exist

I have a form in Access 2010 that's used as a search form to filter records matching specific criteria.
I transferred information in the backend from one set of tables to another. Now, the filter doesn't work. Even if I leave all the criteria blank - ie. set it to bring up all records - it tells me, 'No records found.'
I've remapped the tables a few times, made sure they all have information, and are linking and opening properly. What could be preventing Access from finding the records?
Here's the filter query, if it helps any. It doesn't appear to be filtering properly, even though it works fine with the old tables.
SELECT Activity.*, ActivityCash.*, EngSchDates.*, Monitoring.*, Procurement.*,
LookupDistrict.*
FROM ((((Activity LEFT JOIN LookupDistrict ON Activity.District =
LookupDistrict.District) INNER JOIN ActivityCash ON Activity.GWP = ActivityCash.GWP)
INNER JOIN EngSchDates ON Activity.GWP = EngSchDates.GWP)
INNER JOIN Procurement ON Activity.GWP = Procurement.GWP) INNER JOIN Monitoring ON
Activity.GWP = Monitoring.GWP ORDER BY Activity.District,
Activity.[ProgramYear], [Activity].GWP;
In general, to debug these types of problems, try removing one table at a time from the FROM clause (and SELECT) until you get your results back.
Remove AND [Activity].[Designer] like '*' from the query.

MySQL with two Joins and a Where clause

I have 3 tables I want to join.
OFFICE contains address/contact details for an office.
CRIMECAT contains categories of crime law that an office may deal with and is related to the OFFICE table via the 'f_id'
CIVILCAT contains categories of civil law that an office may deal with and is related to the OFFICE table via the 'f_id' as well.
An office may deal with categories of crime law, civil law, both or none.
The user puts in a location and also decides via a range of checkboxes which areas of law they're interested in before hitting search. This should then return a list of the addresses of offices that deal with any of the checked categories in that location.
This works perfectly for either a crime category or a civil category, but as soon as one or more of each is selected the query returns zero results.
The working style of query is as follows:
SELECT DISTINCT OF.f_id, OF.acc, OF.add1, OF.add2, OF.add3, OF.city, OF.pc, OF.tel
FROM office OF
JOIN crimecat CR ON OF.f_id=CR.f_id
WHERE OF.id ='3946' AND CR.cat = 'crm'
The query I'm banging my head against a brick wall on is:
SELECT DISTINCT OF.f_id, OF.acc, OF.add1, OF.add2, OF.add3, OF.city, OF.pc, OF.tel
FROM office OF
JOIN crimecat CR ON OF.f_id=CR.f_id
JOIN civilcat CI ON OF.f_id=CI.f_id
WHERE OF.id ='3946' AND ((CR.cat = 'crm') OR (CI.cat = 'aap'))
I've also tried using a variant of the WHERE clause which also returns zero:
WHERE (OF.id ='3946' AND CR.cat = 'crm') OR (OF.id ='3946' AND CI.cat = 'aap')
I'm beginning to think the issue is with the JOIN(s) rather than the WHERE clause but can't think of a better way of writing them.
something like this may help. i suspect you are looking for either , but you are requesting both
SELECT DISTINCT OF.f_id, OF.acc, OF.add1, OF.add2, OF.add3, OF.city, OF.pc, OF.tel
FROM office OF
LEFT JOIN crimecat CR ON OF.f_id=CR.f_id
LEFT JOIN civilcat CI ON OF.f_id=CI.f_id
WHERE OF.id ='3946' AND ((CR.cat = 'crm') OR (CI.cat = 'aap'))

Need help on using SQL Query in Crystal Reports on Vb.NET 2010

I'm using MySQL with Vb.Net 2010 and Crystal Reports and to be honest I'm a complete noob when it comes to using Sql Queries. The problem is, I want to generate a report that will show the service fee from a transaction a person made while also showing the items bought, and their quantity and prices, basically a bill. I created a command from the data source I created in crystal reports and it contains this query:
SELECT t.transaction_code,t.fee,t.service_type,t.date_sched,total_bill, s.item_code, s.quantity_sold, i.item_description, i.price, CONCAT(p.fname,' ',p.mi,' ',p.lname),p.address FROM tbl_transaction AS t, tbl_sale AS s, tbl_item AS i, tbl_profile as p WHERE t.transaction_code = '{?trans_code}' AND s.transaction_code = t.transaction_code AND i.item_code = s.item_code AND p.id='{?p_id}'
It all works fine if I use that, the report shows everything like the persons name, transaction code, service type and its fee as well as all the items bought and the total bill. The problem is, if in that certain transaction, the said person only required services and did not buy any items, the report will just be blank. What I want is that the report should still be able to display the person's name, transaction code, service type and its fee, etc., even without buying an item. I know it's kind of hard to understand and it's also hard for me to explain, but I really need help right now.
Since I can't post images yet since I'm a new user. I'll just post a link to where I uploaded the structures of my tables:
http://oi42.tinypic.com/xogw9k.jpg
I also have a table called link_profile_transaction which does what it says and another table for the services which is linked to the services and fee from the transaction table(tbl_transaction) though it isn't included in the query.
http://oi41.tinypic.com/fqm2b.jpg
The links are legit and safe.
You must left join the items table to the service table.
Not really certain about the schema of your table, I am providing below for you to edit if it does not work as is.
SELECT t.transaction_code,t.fee,t.service_type,… s.item_code, s.quantity_sold,
i.item_description, i.price, CONCAT(p.fname,' ',p.mi,' ',p.lname),p.address
FROM tbl_transaction AS t, tbl_sale AS s, tbl_profile as p
LEFT JOIN tbl_item AS i ON i.item_code = s.item_code
WHERE t.transaction_code = '{?trans_code}'
AND s.transaction_code = t.transaction_code AND
p.id='{?p_id}'