We are running a custom app on Invantive Data Access Point which adds business functionality to Exact Online. For billing purposes, we would like to somehow register actual use of the software as defined in business terms instead of memory used, CPU, SQL statements executed, etc.
We do not yet have custom tables and I would like to keep it that way, so the whole state is kept in memory and in Exact Online only. So "insert into mytable#sqlserver..." is not an option. Neither does Exact Online offer the possibility to create custom tables as with Salesforce.
How can we somehow register billable events, such as "Performed an upload of 8 bank transactions" under this condition?
For billing purposes, you can lift along on the Customer Service infrastructure, which is similar to functionality offered by AWS or Apple for this purpose in their eco system. The "table" which stores the billing events like a Call Detail Record of a PBX is managed by Customer Service infrastructure.
There are two options:
Your apps use the default audit and license event registrations like "User logged on", "First use of partition #xyz", etc. each with a specific message code like 'itgenlic125'.
Your apps define their own event types like "Performed an upload of bank transactions", with a message code 'mybillingmessagecode123' and the number '8' as quantity in the natural key.
The first option is automatically and always done. These data is also used to manage resource consumption and detect runaways.
The second option is best done using Invantive SQL with the data dictionary table "auditevents". All records inserted into auditevents are automatically asynchronously forwarded to Customer Service. To see the current register audit events since start of application:
select *
from auditevents#datadictionary
where:
occurrence_date: when it happened.
logging_level: always "Audit".
message_code: code identifying the type of event.
data_container_d: ID of the data container, used with distributed SQL transactions.
partition: partition within the data container for platforms such as Exact Online or Microsoft SQL Server which store multiple databases under one customer/instance.
session_id: ID of the session.
user_message: actual text.
last_nk: last used natural key
application_name: name of the appplication.
application_user: user as known to the application.
gui_action: action within the GUI.
And some auditing and licensing information fields.
To register a custom event:
insert into auditevents#datadictionary select * from auditevents#datadictionary
Only some fields can be provided; the rest are automatically determined:
message_code
user_message
last_natural_key
application_name
application_user
gui_action
gui_module
partition
provider_name
reference_key
reference_table_code
session_id
To receive the billing events yourself from the infrastructure, you will need to access the Customer Service APIs or have them automatically forwarded to mail, Slack, RocketChat or Mattermost channel.
A sample SQL:
insert into auditevents#datadictionary
( message_code
, user_message
, last_natural_key
, application_name
, gui_action
, gui_module
, reference_key
, reference_table_code
, partition
)
select 'xxmycode001' message_code
, 'Processed PayPal payments in Exact Online for ' || divisionlabel user_message
, 'today' last_natural_key
, 'PayPalProcessor' application_name
, 'xx-my-paypal-processor-step-2' gui_action
, 'xx-my-payal-processor' gui_module
, clr_id reference_key
, 'clr' reference_table_code
, division partition
from settings#inmemorystorage
Related
We have a few thousand companies in Exact Online from which a certain percentage runs their own accounting and has their own license. However, there is a daily changing group of companies that are behind with their payments to Exact and therefore their companies are blocked.
For all companies we run Invantive Data Replicator to replicate all Exact Online companies into a SQL Server datawarehouse for analytical reporting and continuous monitoring.
In the SystemDivisions table, the state of such a blocked company remains at 1 (Active). It does not change to 2 (Archive) or 0 (upcoming). Nor is there any enddate set in the past.
However, when the XML or REST APIs are used through a query from Invantive SQL or directly from Python on such a blocked company there are lot of fuzzy error messages.
Currently we have to open each company which had an error during replication individually each day and check whether a block by Exact is causing the error and for what reason.
It seems that there is no way to retrieve the list of blocked companies.
Is there an alternative?
Although it is not supported and disadviced, you can access a limited number of screens in Exact Online using native requests. It is rumoured that this is not possible for all screens.
However, you are lucky. The blocking status of a company can be requested using the following queries:
insert into NativePlatformScalarRequests(url, orig_system_group)
select /*+ ods(false) */ 'https://start.exactonline.nl/docs/SysAccessBlocked.aspx?_Division_=' || code
, 'BLOCK-DIV-CHECK-' || code
from systemdivisions
create or replace table currentlyblockeddivisions#inmemorystorage
as
select blockingstatus
, divisioncode
from ( select regexp_replace(result, '.*<table class="WizardSectionHeader" style="width:100%;"><tr><th colspan="2">([^<]*)</th>.*', '$1', 1, 0, 'n') blockingstatus
, replace(orig_system_group, 'BLOCK-DIV-CHECK-', '') divisioncode
from NativePlatformScalarRequests
where orig_system_group like 'BLOCK-DIV-CHECK-%'
)
where blockingstatus not like '%: Onbekend%'
Please note that the hyperlink with '.nl' needs to be replaced when you run on a different country. The same holds for searching on the Dutch term 'Onbekend' ('Unknown' in english).
This query runs several thousand of HTTP request, each requesting the screen with the blocking status of a company. However, when the company is not blocked, the screen reports back a reason of 'Unknown'.
These companies with 'Unknown' reason are probably not blocked. The rest is.
This query creates an export for UPS from the deliveries history:
select 'key'
, ACC.Name
, CON.FullName
, CON.Phone
, ADR.AddressLine1
, ADR.AddressLine2
, ADR.AddressLine3
, ACC.Postcode
, ADR.City
, ADR.Country
, ACC.Code
, DEL.DeliveryNumber
, CON.Email
, case
when CON.Email is not null
then 'Y'
else 'N'
end
Ship_Not_Option
, 'Y' Ship_Not
, 'ABCDEFG' Description_Goods
, '1' numberofpkgs
, 'PP' billing
, 'CP' pkgstype
, 'ST' service
, '1' weight
, null Shippernr
from ExactOnlineREST..GoodsDeliveries del
join ExactOnlineREST..Accounts acc
on ACC.ID = del.DeliveryAccount
join ExactOnlineREST..Addresses ADR
on ADR.ID = DEL.DeliveryAddress
join ExactOnlineREST..Contacts CON
on CON.ID = DEL.DeliveryContact
where DeliveryDate between $P{P_SHIPDATE_FROM} and $P{P_SHIPDATE_TO}
order
by DEL.DeliveryNumber
It takes many minutes to run. The number of deliveries and accounts grows with several hundreds each day. Addresses and contacts are mostly 1:1 with accounts. How can this query be optimized for speed in Invantive Control for Excel?
Probably this query is run at most once every day, since the deliverydate does not contain time. Therefore, the number of rows selected from ExactOnlineREST..GoodsDeliveries is several hundreds. Based upon the statistics given, the number of accounts, deliveryaddresses and contacts is also approximately several hundreds.
Normally, such a query would be optimized by a solution such as Exact Online query with joins runs more than 15 minutes, but that solution will not work here: the third value of a join_set(soe, orderid, 100) is the maximum number of rows on the left-hand side to be used with index joins. At this moment, the maximum number on the left-hand side is something like 125, based upon constraints on the URL length for OData requests to Exact Online. Please remember the actual OData query is a GET using an URL, not a POST with unlimited size for the filter.
The alternatives are:
Split volume
Data Cache
Data Replicator
Have SQL engine or Exact Online adapted :-)
Split Volume
In a separate query select the eligible GoodsDeliveries and put them in an in-memory or database table using for instance:
create or replace table gdy#inmemorystorage as select ... from ...
Then create a temporary table per 100 or similar rows such as:
create or replace table gdysubpartition1#inmemorystorage as select ... from ... where rowidx$ between 0 and 99
... etc for 100, 200, 300, 400, 500
And then run the query several times, each time with a different gdysubpartition1..gdysubpartition5 instead of the original from ExactOnlineREST..GoodsDeliveries.
Of course, you can also avoid the use of intermediate tables by using an inline view like:
from (select * from goodsdeliveries where date... limit 100)
or alike.
Data Cache
When you run the query multiple times per day (unlikely, but I don't know), you might want to cache the Accounts in a relational database and update it every day.
You can also use a 'local memorize results clipboard andlocal save results clipboard to to save the last results to a file manually and later restore them usinglocal load results clipboard from ...andlocal insert results clipboard in table . And maybe theninsert into from exactonlinerest..accounts where datecreated > trunc(sysdate)`.
Data Replicator
With Data Replicator enabled, you can have replicas created and maintained automatically within an on-premise or cloud relational database for Exact Online API entities. For low latency, you will need to enable the Exact webhooks.
Have SQL Engine or Exact adapted
You can also register a request to have the SQL engine to allow higher number in the join_set hint, which would require addressing the EOL APIs in another way. Or register a request at Exact to also allow POST requests to the API with the filter in the body.
The SQL engine hides away all nifty details on what API calls are being done. However, some cloud solutions have pricing per API call.
For instance:
select *
from transactionlines
retrieves all Exact Online transaction lines of the current company, but:
select *
from transactionlines
where financialyear = 2016
filters it effectively on REST API of Exact Online to just that year, reducing data volume. And:
select *
from gltransactionlines
where year_attr = 2016
retrieves all data since the where-clause is not forwarded to this XML API of Exact.
Of course I can attach fiddler or wireshark and try to analyze the data volume, but is there an easier way to analyze the data volume of API calls with Invantive SQL?
First of all, all calls handled by Invantive SQL are logged in the Invantive Cloud together with:
the time
data volume in both directions
duration
to enable consistent API use monitoring across all supported cloud platforms. The actual data is not logged and travels directly.
You can query the same numbers from within your session, for instance:
select * from exactonlinerest..projects where code like 'A%'
retrieves all projects with a code starting with 'A'. And then:
select * from sessionios#datadictionary
shows you the API calls made:
You can also put a query like to following at the end of your session before logging off:
select main_url
, sum(bytes_received) bytes_received
, sum(duration_ms) duration_ms
from ( select regexp_replace(url, '\?.*', '') main_url
, bytes_received
, duration_ms
from sessionios#datadictionary
)
group
by main_url
with a result such as:
I am new to selenium webdriver python . I need to automate back end process of an e-commerce application ( order processing)
here the orders are classified to three types :
say A-type , B-type and C-type .
After successful order completion the orders will listed in an interface .
In all listed orders, its order type will mentioned like
"ORDER TYPE:A-type"
According to these three types certain scenarios will be executed. I need to identify this type. In a page there may be more than one A-type/B-type/C-type orders.
here the orders will be listing one by one will all user details along with order type. please help in this.
its html tag details:
order Type: A-type
I'm really beginner of nodejs. I want to make a chatting service using nodejs. I use nodejs/jade/mysql to construct basic part of my system and now I want to provide pub/sub to users.
We receive users' interests from text field or using hash tags (anyway we received users' interests and stored in MySQL -> we did it). Then, we want to show users chatting room list according to their interests. For instance A's interests are 'game', 'car' and 'food', then we search chat rooms with 'game', 'car', 'food' and show A these chat rooms first.
I want to use redis to provide this service but i really have no idea!
1) I installed redis and can run redis-server.
2)
//redis
var redis = require('redis');
var publisher = redis.createClient();
var subscriber = redis.createClient();
subscriber.on('message', function(channel, message){
console.log('Message ' + message + ' on channel ' + channel + ' arrived!');
});
subscriber.on('subscribe', function(channel){
publisher.publish('test', 'the a team');
publisher.publish('test', 'the b team');
})
subscriber.subscribe('test');
This is short code that I tried to understand redis.
3) I don't know how can I read data stored in Mysql and show users chat room according to their interests using redis.
Redis is a advanced key-value cache and store.Its operations cannot be directly mapped to mysql.
In redis you can set either key value pair or a hash under a key.
That is :
If you want to store your name in redis it can be done by:
var client = redis.createClient();
client.set("name", "John")
Retrieve the values using client.get("name")
Similarly under a single key you can store multiple key value pairs, as hash.
That under a name if you want to store their details like age, place, company etc.Then hash should be used.
Redis has method "hmset" and "hmget" for hash opertaions.
In redis like in cache you can set expiry time.
There are different method available. You can explore those.
For reference http://redis.io/commands