Related
JMeter 5.3
I use the CLI as follows:
C:\Users\guyl\OneDrive - xxxxLTD\Guy\apache-jmeter-5.3\bin>jmeter -n -t "C:\Users\guyl\OneDrive - xxxxLTD\Guy\JMeter\DCS DB threaded test.jmx" -l"C:\Users\guyl\OneDrive - xxxxLTD\Guy\JMeter\db_report.csv" -Jthreads=5 -Jloops=100 -e -o"C:\Users\guyl\OneDrive - xxxxLTD\Guy\JMeter\output\19082020\"
Creating summariser <summary>
Created the tree successfully using C:\Users\guyl\OneDrive - xxxLTD\Guy\JMeter\DCS DB threaded test.jmx
Starting standalone test # Wed Aug 26 14:22:56 IDT 2020 (1598440976507)
Waiting for possible Shutdown/StopTestNow/HeapDump/ThreadDump message on port 4445
Generate Summary Results + 91 in 00:00:03 = 32.0/s Avg: 35 Min: 0 Max: 1031 Err: 0 (0.00%) Active: 2 Started: 2 Finished: 0
summary + 91 in 00:00:03 = 32.1/s Avg: 35 Min: 0 Max: 1031 Err: 0 (0.00%) Active: 2 Started: 2 Finished: 0
summary + 104409 in 00:00:29 = 3621.4/s Avg: 1 Min: 0 Max: 448 Err: 50400 (48.27%) Active: 0 Started: 5 Finished: 5
summary = 104500 in 00:00:32 = 3299.8/s Avg: 1 Min: 0 Max: 1031 Err: 50400 (48.23%)
Generate Summary Results + 104409 in 00:00:29 = 3620.9/s Avg: 1 Min: 0 Max: 448 Err: 50400 (48.27%) Active: 0 Started: 5 Finished: 5
Generate Summary Results = 104500 in 00:00:32 = 3299.1/s Avg: 1 Min: 0 Max: 1031 Err: 50400 (48.23%)
Tidying up ... # Wed Aug 26 14:23:28 IDT 2020 (1598441008843)
Error generating the report: org.apache.jmeter.report.dashboard.GenerationException: Data exporter "json" is unable to export data.
... end of run
At the end, you can see we have:
Error generating the report: org.apache.jmeter.report.dashboard.GenerationException: Data exporter "json" is unable to export data.
... end of run
Note that it didn't matter whether the report's extension was CSV or JTL.
I was able to generate the JTL report, and then run jmeter -g <my JTL file>, but I'd like the -e option to work.
Update: Now I get errors with the g option:
C:\Users\guyl\OneDrive - xxxLTD\Guy\apache-jmeter-5.3\bin>jmeter -g "C:\Users\guyl\OneDrive - xxxLTD\Guy\JMeter\db_report" -o"C:\Users\guyl\OneDrive - xxxLTD\Guy\JMeter\output\19082020\"
An error occurred: Data exporter "json" is unable to export data.
errorlevel=1
Press any key to continue . . .
Here is what I found in the jmeter.log file:
2020-08-26 14:37:51,812 INFO o.a.j.r.p.AbstractSampleConsumer: class org.apache.jmeter.report.processor.FilterConsumer#stopProducing(): nameFilter produced 1567500 samples
2020-08-26 14:37:51,812 INFO o.a.j.r.p.AbstractSampleConsumer: class org.apache.jmeter.report.processor.FilterConsumer#stopProducing(): dateRangeFilter produced 313500 samples
2020-08-26 14:37:51,812 INFO o.a.j.r.p.AbstractSampleConsumer: class org.apache.jmeter.report.processor.NormalizerSampleConsumer#stopProducing(): normalizer produced 104500 samples
2020-08-26 14:37:51,813 INFO o.a.j.r.p.CsvFileSampleSource: produce(): 104500 samples produced in 6s 70 ms on channel 0
2020-08-26 14:37:51,813 INFO o.a.j.r.d.ReportGenerator: Exporting data using exporter:'json' of className:'org.apache.jmeter.report.dashboard.JsonExporter'
2020-08-26 14:37:51,814 INFO o.a.j.r.d.JsonExporter: Found data for consumer statisticsSummary in context
2020-08-26 14:37:51,814 INFO o.a.j.r.d.JsonExporter: Creating statistics for overall
2020-08-26 14:37:51,815 INFO o.a.j.r.d.JsonExporter: Creating statistics for other transactions
2020-08-26 14:37:51,815 INFO o.a.j.r.d.JsonExporter: Checking output folder
2020-08-26 14:37:51,816 ERROR o.a.j.JMeter: An error occurred:
org.apache.jmeter.report.dashboard.GenerationException: Data exporter "json" is unable to export data.
at org.apache.jmeter.report.dashboard.ReportGenerator.exportData(ReportGenerator.java:385) ~[ApacheJMeter_core.jar:5.3]
at org.apache.jmeter.report.dashboard.ReportGenerator.generate(ReportGenerator.java:258) ~[ApacheJMeter_core.jar:5.3]
at org.apache.jmeter.JMeter.start(JMeter.java:545) [ApacheJMeter_core.jar:5.3]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_241]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:1.8.0_241]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:1.8.0_241]
at java.lang.reflect.Method.invoke(Unknown Source) ~[?:1.8.0_241]
at org.apache.jmeter.NewDriver.main(NewDriver.java:252) [ApacheJMeter.jar:5.3]
Caused by: org.apache.jmeter.report.dashboard.ExportException: Error creating output folder C:\Users\guyl\OneDrive - Nayax LTD\Guy\JMeter\output\19082020"
at org.apache.jmeter.report.dashboard.JsonExporter.checkAndGetOutputFolder(JsonExporter.java:112) ~[ApacheJMeter_core.jar:5.3]
at org.apache.jmeter.report.dashboard.JsonExporter.export(JsonExporter.java:77) ~[ApacheJMeter_core.jar:5.3]
at org.apache.jmeter.report.dashboard.ReportGenerator.exportData(ReportGenerator.java:379) ~[ApacheJMeter_core.jar:5.3]
... 7 more
Caused by: java.io.IOException: Unable to create directory C:\Users\guyl\OneDrive - xxxLTD\Guy\JMeter\output\19082020"
at org.apache.commons.io.FileUtils.forceMkdir(FileUtils.java:2491) ~[commons-io-2.6.jar:2.6]
at org.apache.jmeter.report.dashboard.JsonExporter.checkAndGetOutputFolder(JsonExporter.java:110) ~[ApacheJMeter_core.jar:5.3]
at org.apache.jmeter.report.dashboard.JsonExporter.export(JsonExporter.java:77) ~[ApacheJMeter_core.jar:5.3]
at org.apache.jmeter.report.dashboard.ReportGenerator.exportData(ReportGenerator.java:379) ~[ApacheJMeter_core.jar:5.3]
... 7 more
So it all started from:
Caused by: java.io.IOException: Unable to create directory C:\Users\guyl\OneDrive - xxx LTD\Guy\JMeter\output\19082020"
It's supposed to create that folder if it doesn't exist, isn't it?
JMeter will create only 1 level of folder and not the full hierarchy.
Second , try avoiding a folder with spaces in it.
I was getting this error as well and the solution to my problem was that from this command:
jmeter -n -t “location of test file” -l “location of your result file” -e -o “location of reports folder”
The third option "location of reports folder" needed to be a brand new non-existing folder. After telling the command to create a folder, my HTML reports were successful!
I've just experienced the same error message, using jMeter 5.4.1.
In my case, I ended up investigating the problem using ProcMon
Procmon showed that the closing double quote " was included in the folder path, which is an invalid character in windows folder/file names.
I was fortunate enough that my path did not have any spaces in it so removing the double quote did not impact me. Try to use folders without spaces in the names, or alternatively use the 8.3 short names in the paths instead.
Currently, I am trying to update the JSON response node value using BeanShell PreProcessor.
Need help in converting JSON response in Object and then updating the node value and then stringify.
I have the following JSON and I want to update the OrderTypeId and IsCredentialOff values using BeanShell PreProcessor in Jmeter. I have tried a solution in postman which worked well. whereas getting difficulty in BeanShell Scripting.
{"DeliveryCharge":0.0,"ApplyCouponToCart":true,"CartId":6970,"SessionId":"DB84A55E-82A9-4BC9-98E8-2A845FB6893A","StoreId":10061,"UserId":10011690,"AppId":10061,"ListCartItem":[{"CartItemId":55513,"UserId":0,"StoreId":0,"AppId":0,"CartId":6970,"PID":772544,"Pack":1,"ProductId":5712,"ProductImage":"https://s3.amazonaws.com/images.liquorapps.com/sm/000457-TWO-FINGERS-GOLD-TEQUILA.png","ProductName":"TWO FINGERS GOLD","UnitSize":"750 ML","Quantity":1,"UPC":"89540135328","SKU":"1014","CategoryId":2,"TypeId":15,"VarietalId":0,"UnitSizeId":12,"CountryId":7,"RegionId":32,"Price":14.99,"PriceDisplay":"$14.99","OfferPrice":14.99,"OfferPriceDisplay":"$14.99","FinalPrice":14.99,"FinalPriceDisplay":"$14.99","TaxRate":0.066250,"QuantityOrdered":1,"Remark":"","ItemTotal":14.99,"ItemTotalDisplay":"$14.99","FinalItemTotal":14.99,"FinalItemTotalDisplay":"$14.99","ItemTotalSaving":0.00,"ItemTotalSavingDisplay":"","ListDiscount":[],"ListCharge":[],"CartItemCount":0,"Deposit":0.00,"DealId":0,"DealInventory":0,"DealInventoryMessage":"","IsBottleLimitAtRetail":false,"TotalQuantity":1,"IsOfferItem":false,"SuccessMessage":"","ErrorDetail":"","ErrorMessage":"","MessageType":"","MessageTitle":"","TextType":""}],"ListDiscount":[],"ListCharge":[],"CouponCode":"","OrderTypeId":1,"AddressId":0,"PaymentTypeId":0,"TipForDriver":0.00,"ListTipForDriver":[],"SubTotal":14.99,"SubTotalDisplay":"$14.99","TotalSavings":0.0,"TotalSavingsDisplay":"-$0.00","TotalCharges":0.0,"TotalChargesDisplay":"$0.00","TotalValue":14.99,"TotalValueDisplay":"$14.99","SubTotalAfterDsicount":14.99,"SubTotalAfterDiscountDisplay":"$14.99","CartPaymentItemUser":{"UserId":10011690,"StoreId":10061,"AppId":0,"PaymentTypeId":0,"UserProfileId":"","IsDefault":false,"IsCardDefault":false,"Credential1":"","Credential2":"UdVYyaeBH+IBzsetzftdBw==","Credential3":"","IsLive":false},"DoPDate":"01/01/1900","DoPTimeSlot":"","DopUtcStartDate":"","DopUtcEndDate":"","Remark":"","DeliveryTax":0.0,"ShippingCost":0.0,"CartItemCount":1,"IsCredentialOff":false,"IsDeliverySolution":false,"DeliveryFeeMarkUp":0.0,"ShippingTax":0.0,"DeliveryTaxRate":0.0,"IsFromCheckOut":true,"IsUpgradeUser":true,"IsToCallDSP":false,"DeliveryInstruction":"","IsStoreCreditUtilize":false,"OrderRewardCredit":5.00,"OrderRewardCreditDisplay":"-$5.00","OrderTotalWithCredit":0.0,"IsStoreReward":false,"CouponRemark":"","IsShipping":false,"CartTotalQuan":1,"ShippingServiceCode":"","ActualDeliveryCharge":0.0,"EstimatedShippingCost":0.0,"ShippingDeliveryCharge":0.0,"DoordashValidateCustomer":false,"CustomerSessionId":13733,"ConvenienceFee":0.0,"IsDoCalculation":false,"SuccessMessage":"","ErrorDetail":"","ErrorMessage":"","MessageType":"","MessageTitle":"","TextType":""}
I have tried in BeanShell PreProcessor:
var CartGetDetailsObject = prev.getResponseDataAsString();
var jsonData = JSON.parse(CartGetDetailsObject);
jsonData.OrderTypeId=2;
jsonData.IsCredentialOff=true;
vars.put("vCartGetDetailsObject",JSON.stringify(jsonData));
log.info(vCartGetDetailsObject);
Error Log
2020-04-06 12:56:48,139 INFO o.a.j.e.StandardJMeterEngine: Running the test!
2020-04-06 12:56:48,145 INFO o.a.j.s.SampleEvent: List of sample_variables: []
2020-04-06 12:56:48,153 INFO o.a.j.r.ResultCollector: Folder at F:\Office\Performance Test Cases\BottlecappsJmeterPerformanceScripts\apache-jmeter-5.0\RunScript\Date 6-Apr-2020 125648 was created
2020-04-06 12:56:48,154 INFO o.a.j.g.u.JMeterMenuBar: setRunning(true, *local*)
2020-04-06 12:56:48,958 INFO o.a.j.e.StandardJMeterEngine: Starting ThreadGroup: 1 : Thread Group
2020-04-06 12:56:48,958 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group Thread Group.
2020-04-06 12:56:48,958 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error
2020-04-06 12:56:48,958 INFO o.a.j.t.ThreadGroup: Starting thread group... number=1 threads=1 ramp-up=0 perThread=0.0 delayedStart=false
2020-04-06 12:56:48,984 INFO o.a.j.t.ThreadGroup: Started thread group number 1
2020-04-06 12:56:48,984 INFO o.a.j.e.StandardJMeterEngine: All thread groups have been started
2020-04-06 12:56:48,986 INFO o.a.j.t.JMeterThread: Thread started: Thread Group 1-1
2020-04-06 12:56:48,987 INFO o.a.j.s.FileServer: Stored: F:\Office\Performance Test Cases\BottlecappsJmeterPerformanceScripts\apache-jmeter-5.0\testrundata.csv
2020-04-06 12:56:53,982 ERROR o.a.j.u.BeanShellInterpreter: Error invoking bsh method: eval Sourced file: inline evaluation of: ``var CartGetDetailsObject = prev.getResponseDataAsString(); var jsonData = JSON. . . . '' : Typed variable declaration : Attempt to resolve method: parse() on undefined variable or class name: JSON
2020-04-06 12:56:53,982 WARN o.a.j.m.BeanShellPreProcessor: Problem in BeanShell script. org.apache.jorphan.util.JMeterException: Error invoking bsh method: eval Sourced file: inline evaluation of: ``var CartGetDetailsObject = prev.getResponseDataAsString(); var jsonData = JSON. . . . '' : Typed variable declaration : Attempt to resolve method: parse() on undefined variable or class name: JSON
2020-04-06 12:56:54,645 ERROR o.a.j.p.h.s.HTTPJavaImpl: readResponse: java.io.IOException: Server returned HTTP response code: 400 for URL: https://staging.liquorapps.com/Bcapi/api/Cart/CartUpdate
2020-04-06 12:56:54,645 ERROR o.a.j.p.h.s.HTTPJavaImpl: Cause: java.io.IOException: Server returned HTTP response code: 400 for URL: https://staging.liquorapps.com/Bcapi/api/Cart/CartUpdate
2020-04-06 12:56:54,645 INFO o.a.j.p.h.s.HTTPJavaImpl: Error Response Code: 400
2020-04-06 12:56:57,171 ERROR o.a.j.u.BeanShellInterpreter: Error invoking bsh method: eval Sourced file: inline evaluation of: ``var CartGetDetailsObject = prev.getResponseDataAsString(); var jsonData = JSON. . . . '' : Typed variable declaration : Attempt to resolve method: parse() on undefined variable or class name: JSON
2020-04-06 12:56:57,171 WARN o.a.j.m.BeanShellPreProcessor: Problem in BeanShell script. org.apache.jorphan.util.JMeterException: Error invoking bsh method: eval Sourced file: inline evaluation of: ``var CartGetDetailsObject = prev.getResponseDataAsString(); var jsonData = JSON. . . . '' : Typed variable declaration : Attempt to resolve method: parse() on undefined variable or class name: JSON
2020-04-06 12:56:57,389 ERROR o.a.j.p.h.s.HTTPJavaImpl: readResponse: java.io.IOException: Server returned HTTP response code: 400 for URL: https://staging.liquorapps.com/Bcapi/api/Cart/CartUpdate
2020-04-06 12:56:57,389 ERROR o.a.j.p.h.s.HTTPJavaImpl: Cause: java.io.IOException: Server returned HTTP response code: 400 for URL: https://staging.liquorapps.com/Bcapi/api/Cart/CartUpdate
2020-04-06 12:56:57,389 INFO o.a.j.p.h.s.HTTPJavaImpl: Error Response Code: 400
2020-04-06 12:56:58,364 INFO o.a.j.g.a.Start: Stopping test
2020-04-06 12:56:58,377 INFO o.a.j.t.JMeterThread: Stopping: Thread Group 1-1
2020-04-06 12:56:58,377 WARN o.a.j.t.JMeterThread: Interrupting: Thread Group 1-1 sampler: LoginCustomer
2020-04-06 12:56:58,382 INFO o.a.j.t.JMeterThread: Thread finished: Thread Group 1-1
2020-04-06 12:56:58,383 INFO o.a.j.e.StandardJMeterEngine: Notifying test listeners of end of test
2020-04-06 12:56:58,383 INFO o.a.j.s.FileServer: Close: F:\Office\Performance Test Cases\BottlecappsJmeterPerformanceScripts\apache-jmeter-5.0\testrundata.csv
2020-04-06 12:56:58,385 INFO o.a.j.g.u.JMeterMenuBar: setRunning(false, *local*)
JSON is a standard built-in object for Web API, you can run it in Postman because Postman is an electron application (browser). JMeter isn't a browser therefore you cannot use JSON functions.
Add JSR223 PostProcessor as a child of the request which returns above JSON
Put the following code into "Script" area:
def jsonData = new groovy.json.JsonSlurper().parse(prev.getResponseData())
jsonData.OrderTypeId = 2
jsonData.IsCredentialOff = true
vars.put("vCartGetDetailsObject", new groovy.json.JsonBuilder(jsonData).toPrettyString())
log.info(vars.get('vCartGetDetailsObject'))
More information:
Apache Groovy - Parsing and producing JSON
Apache Groovy - Why and How You Should Use It
We would like to execute a package comparer to compare Chocolatey packages from the local repository with the public repository. Now when we execute the playbook, the copy of the script goes well but the execution of the script goes good too but then fails for some reason. Maybe one of you could help me with this?
tasks:
- name: Copy powershell file to Chocolatey server
win_copy:
src: ../powershell/package_comparer.ps1
dest: C:\Temp\
- name: Executing Powershell script
win_shell: C:\Temp\package_comparer.ps1
changed_when: false
register: result
- name: parse .json file
set_fact:
packages_result: "{{(result.stdout | from_json)}}"
- debug:
msg: "{{ packages_results }}"
That we can execute this without error.. As I get now this error:
[WARNING]: Failure using method (v2_runner_on_failed) in callback plugin (<ansible.plugins.callback.yaml.CallbackModule object at 0x7fecd4ee8910>): value must be a string
Callback Exception:
File "/usr/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 333, in send_callback
method(*new_args, **kwargs)
File "/usr/lib/python2.7/site-packages/ansible/plugins/callback/default.py", line 93, in v2_runner_on_failed
self._display.display("fatal: [%s]: FAILED! => %s" % (result._host.get_name(), self._dump_results(result._result)),
File "/usr/lib/python2.7/site-packages/ansible/plugins/callback/yaml.py", line 123, in _dump_results
dumped += to_text(yaml.dump(abridged_result, allow_unicode=True, width=1000, Dumper=AnsibleDumper, default_flow_style=False))
File "/usr/lib64/python2.7/site-packages/yaml/__init__.py", line 293, in dump
return dump_all([data], stream, Dumper=Dumper, **kwds)
File "/usr/lib64/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all
dumper.represent(data)
File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 30, in represent
self.serialize(node)
File "_yaml.pyx", line 1348, in _yaml.CEmitter.serialize (ext/_yaml.c:15963)
File "_yaml.pyx", line 1510, in _yaml.CEmitter._serialize_node (ext/_yaml.c:18037)
File "_yaml.pyx", line 1431, in _yaml.CEmitter._serialize_node (ext/_yaml.c:17021)
PLAY RECAP **********************************************************************************************************************************************************************************************
host_machine : ok=1 changed=1 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
I am setting up a cluster. I tried to join 3 nodes but while re-balancing. I got below error. So i extracted some info from debug.log and unable to identify the exact issue. Appreciate any help.
=========================CRASH REPORT=========================
crasher:
initial call: service_agent:-spawn_connection_waiter/2-fun-0-/0
pid: <0.18486.7>
registered_name: []
exception exit: {no_connection,"index-service_api"}
in function service_agent:wait_for_connection_loop/3 (src/service_agent.erl, line 305)
ancestors: ['service_agent-index',service_agent_children_sup,
service_agent_sup,ns_server_sup,ns_server_nodes_sup,
<0.170.0>,ns_server_cluster_sup,<0.89.0>]
messages: []
links: [<0.18481.7>,<0.18490.7>]
dictionary: []
trap_exit: false
status: running
heap_size: 987
stack_size: 27
reductions: 1195
neighbours:
[ns_server:error,2018-02-12T13:54:43.531-05:00,ns_1#xuodf9.firebrand.com:service_agent-index<0.18481.7>:service_agent:terminate:264]Terminating abnormally
[ns_server:debug,2018-02-12T13:54:43.531-05:00,ns_1#xuodf9.firebrand.com:<0.18487.7>:ns_pubsub:do_subscribe_link:145]Parent process of subscription {ns_config_events,<0.18481.7>} exited with reason {linked_process_died,
<0.18486.7>,
{no_connection,
"index-service_api"}}
[error_logger:error,2018-02-12T13:54:43.531-05:00,ns_1#xuodf9.firebrand.com:error_logger<0.6.0>:ale_error_logger_handler:do_log:203]** Generic server 'service_agent-index' terminating
** Last message in was {'EXIT',<0.18486.7>,
{no_connection,"index-service_api"}}
** When Server state == {state,index,
{dict,6,16,16,8,80,48,
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
{{[[{uuid,<<"55a14ec6b06d72205b3cd956e6de60e7">>}|
'ns_1#xuodf7.firebrand.com']],
[],
[[{uuid,<<"c5e67322a74826bef8edf27d51de3257">>}|
'ns_1#xuodf8.firebrand.com']],
[],
[[{uuid,<<"3b55f7739e3fe85127dcf857a5819bdf">>}|
'ns_1#xuodf9.firebrand.com']],
[],
[[{node,'ns_1#xuodf7.firebrand.com'}|
<<"55a14ec6b06d72205b3cd956e6de60e7">>],
[{node,'ns_1#xuodf8.firebrand.com'}|
<<"c5e67322a74826bef8edf27d51de3257">>],
[{node,'ns_1#xuodf9.firebrand.com'}|
<<"3b55f7739e3fe85127dcf857a5819bdf">>]],
[],[],[],[],[],[],[],[],[]}}},
undefined,undefined,<0.18626.7>,#Ref<0.0.5.56873>,
<0.18639.7>,
{[{<0.18646.7>,#Ref<0.0.5.56891>}],[]},
undefined,undefined,undefined,undefined,undefined}
** Reason for termination ==
** {linked_process_died,<0.18486.7>,{no_connection,"index-service_api"}}
[error_logger:error,2018-02-12T13:54:43.532-05:00,ns_1#xuodf9.firebrand.com:error_logger<0.6.0>:ale_error_logger_handler:do_log:203]
=========================CRASH REPORT=========================
crasher:
initial call: service_agent:init/1
pid: <0.18481.7>
registered_name: 'service_agent-index'
exception exit: {linked_process_died,<0.18486.7>,
{no_connection,"index-service_api"}}
in function gen_server:terminate/6 (gen_server.erl, line 744)
ancestors: [service_agent_children_sup,service_agent_sup,ns_server_sup,
ns_server_nodes_sup,<0.170.0>,ns_server_cluster_sup,
<0.89.0>]
messages: [{'EXIT',<0.18639.7>,
{linked_process_died,<0.18486.7>,
{no_connection,"index-service_api"}}}]
links: [<0.18487.7>,<0.4805.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 28690
stack_size: 27
reductions: 6334
neighbours:
[error_logger:error,2018-02-12T13:54:43.533-05:00,ns_1#xuodf9.firebrand.com:error_logger<0.6.0>:ale_error_logger_handler:do_log:203]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,service_agent_children_sup}
Context: child_terminated
Reason: {linked_process_died,<0.18486.7>,
{no_connection,"index-service_api"}}
Offender: [{pid,<0.18481.7>},
{name,{service_agent,index}},
{mfargs,{service_agent,start_link,[index]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]
[ns_server:error,2018-02-12T13:54:43.533-05:00,ns_1#xuodf9.firebrand.com:service_rebalancer-index<0.18626.7>:service_rebalancer:run_rebalance:80]Agent terminated during the rebalance: {'DOWN',#Ref<0.0.5.56860>,process,
<0.18481.7>,
{linked_process_died,<0.18486.7>,
{no_connection,"index-service_api"}}}
[error_logger:info,2018-02-12T13:54:43.534-05:00,ns_1#xuodf9.firebrand.com:error_logger<0.6.0>:ale_error_logger_handler:do_log:203]
=========================PROGRESS REPORT=========================
supervisor: {local,service_agent_children_sup}
started: [{pid,<0.20369.7>},
{name,{service_agent,index}},
{mfargs,{service_agent,start_link,[index]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]
[ns_server:error,2018-02-12T13:54:43.534-05:00,ns_1#xuodf9.firebrand.com:service_agent-index<0.20369.7>:service_agent:handle_call:186]Got rebalance-only call {if_rebalance,<0.18626.7>,unset_rebalancer} that doesn't match rebalancer pid undefined
[ns_server:error,2018-02-12T13:54:43.534-05:00,ns_1#xuodf9.firebrand.com:service_rebalancer-index<0.18626.7>:service_agent:process_bad_results:815]Service call unset_rebalancer (service index) failed on some nodes:
[{'ns_1#xuodf9.firebrand.com',nack}]
[ns_server:warn,2018-02-12T13:54:43.534-05:00,ns_1#xuodf9.firebrand.com:service_rebalancer-index<0.18626.7>:service_rebalancer:run_rebalance:89]Failed to unset rebalancer on some nodes:
{error,{bad_nodes,index,unset_rebalancer,
[{'ns_1#xuodf9.firebrand.com',nack}]}}
[error_logger:error,2018-02-12T13:54:43.535-05:00,ns_1#xuodf9.firebrand.com:error_logger<0.6.0>:ale_error_logger_handler:do_log:203]
=========================CRASH REPORT=========================
crasher:
initial call: service_rebalancer:-spawn_monitor/6-fun-0-/0
pid: <0.18626.7>
registered_name: 'service_rebalancer-index'
exception exit: {linked_process_died,<0.18486.7>,
{no_connection,"index-service_api"}}
in function service_rebalancer:run_rebalance/7 (src/service_rebalancer.erl, line 92)
ancestors: [cleanup_process,ns_janitor_server,ns_orchestrator_child_sup,
ns_orchestrator_sup,mb_master_sup,mb_master,<0.4893.0>,
ns_server_sup,ns_server_nodes_sup,<0.170.0>,
ns_server_cluster_sup,<0.89.0>]
messages: [{'EXIT',<0.18640.7>,
{linked_process_died,<0.18486.7>,
{no_connection,"index-service_api"}}}]
links: []
dictionary: []
trap_exit: true
status: running
heap_size: 2586
stack_size: 27
reductions: 6359
neighbours:
[ns_server:error,2018-02-12T13:54:43.536-05:00,ns_1#xuodf9.firebrand.com:cleanup_process<0.18625.7>:service_janitor:maybe_init_topology_aware_service:84]Initial rebalance for `index` failed: {error,
{initial_rebalance_failed,index,
{linked_process_died,<0.18486.7>,
{no_connection,
"index-service_api"}}}}
[ns_server:debug,2018-02-12T13:54:43.536-05:00,ns_1#xuodf9.firebrand.com:menelaus_cbauth<0.4796.0>:menelaus_cbauth:handle_cast:95]Observed json rpc process {"projector-cbauth",<0.5099.0>} needs_update
[ns_server:debug,2018-02-12T13:54:43.538-05:00,ns_1#xuodf9.firebrand.com:menelaus_cbauth<0.4796.0>:menelaus_cbauth:handle_cast:95]Observed json rpc process {"goxdcr-cbauth",<0.479.0>} needs_update
[ns_server:debug,2018-02-12T13:54:43.539-05:00,ns_1#xuodf9.firebrand.com:menelaus_cbauth<0.4796.0>:menelaus_cbauth:handle_cast:95]Observed json rpc process {"cbq-engine-cbauth",<0.5124.0>} needs_update
[ns_server:debug,2018-02-12T13:54:43.540-05:00,ns_1#xuodf9.firebrand.com:menelaus_cbauth<0.4796.0>:menelaus_cbauth:handle_cast:95]Observed json rpc process {"fts-cbauth",<0.5129.0>} needs_update
This is a blocker for cluster creation at this point.
The rebalance error is coming due to index service. You can check indexer.log to see if there are any errors and the process is able to bootstrap correctly.
Please make sure the communication ports are open as mentioned here: https://developer.couchbase.com/documentation/server/current/install/install-ports.html
projector_port 9999 being blocked can lead to this.
I'm trying to configure a datasource in IBM WebSphere Liberty Profile (16.0.0.3) and this is what I've done so far:
server.xml
<authData id="dbuser" password="{xor}blablabla" user="MY_USER"/>
<dataSource id="Oracle" isolationLevel="TRANSACTION_READ_COMMITTED"
jdbcDriverRef="OracleDriver"
jndiName="EPMS_DS"
recoveryAuthDataRef="dbuser"
type="javax.sql.ConnectionPoolDataSource">
<properties.oracle databaseName="DBNAME" portNumber="1521" serverName="SERVERNAME"/>
</dataSource>
<jdbcDriver id="OracleDriver"
javax.sql.ConnectionPoolDataSource="oracle.jdbc.pool.OracleConnectionPoolDataSource"
libraryRef="shared-library"/>
web.xml
<resource-env-ref>
<description>The Oracle DS</description>
<resource-env-ref-name>jdbc/OracleDS</resource-env-ref-name>
<resource-env-ref-type>javax.sql.DataSource</resource-env-ref-type>
</resource-env-ref>
ibm-web-bnd.xml
<resource-ref name="jdbc/OracleDS" binding-name="EPMS_DS">
<authentication-alias name="dbuser" />
</resource-ref>
However, besides the application server is taking more than 2 minutes to startup, my application seems to freeze on the following instruction:
ctx = new InitialContext();
ctx.lookup("java:comp/env/jdbc/OracleDS");
The log doesn't show any errors, the last line it shows is an application's debug message indicating it is going to do a JNDI lookup.
I've also tried different configurations in server.xml, without <authData> and explicitly defining user and password on the datasource, but with identical results:
<dataSource id="Oracle" isolationLevel="TRANSACTION_READ_COMMITTED" jdbcDriverRef="OracleDriver" jndiName="EPMS_DS" type="javax.sql.ConnectionPoolDataSource">
<properties.oracle URL="jdbc:oracle:thin:#SERVERNAME:1521:DBNAME" password="{xor}blablabla" user="MY_USER"/>
</dataSource>
Sadly, Liberty Profile doesn't seem to provide a way to test the DB connection, but everything seems correctly configured (I can assure the credentials are correct, as well as the server name and port). What am I missing here?
EDIT #1
Following njr's suggestion, I've performed a thread dump and here is a summary:
- waiting on com.ibm.tx.jta.impl.EventSemaphore#737eaefc
- waiting on com.ibm.ws.objectManager.FileLogOutput$FlushHelper#19d51071
- waiting on com.ibm.ws.objectManager.FileLogOutput$NotifyHelper#2fa0da91
- waiting on com.ibm.ws.objectManager.ObjectManagerState$CheckpointHelper#5b0919fc
- waiting on com.ibm.ws.sib.msgstore.persistence.dispatcher.SpillDispatcher$DispatchingLock#1620db94
(8 Occorrences, but different instances)
...
- waiting on com.ibm.ws.threading.internal.BoundedBuffer$GetQueueLock#c8a05b6
(56 Occorrences, but different instances)
...
- waiting on java.lang.Object#4c1d5897
- waiting on java.lang.ref.Reference$Lock#5448da4c
- waiting on java.lang.ref.ReferenceQueue$Lock#f91b025
- waiting on java.util.LinkedList#5b213416
- waiting on java.util.LinkedList#6cb46e1f
- waiting on java.util.TaskQueue#f50561c
(14 Occorrences, but different instances)
...
- waiting on java.util.concurrent.atomic.AtomicReference#5476d077
- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#4da17c93
- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#513339c6
- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#5dc2ae0f
- waiting on org.eclipse.osgi.framework.eventmgr.EventManager$EventThread#236970be
- waiting on org.eclipse.osgi.framework.eventmgr.EventManager$EventThread#6dfdd5
- waiting on org.eclipse.osgi.framework.eventmgr.EventManager$EventThread#72ce4e1c
- blocked on com.ibm.tx.jta.embeddable.impl.EmbeddableTMHelper#5748c911
- blocked on com.ibm.tx.jta.embeddable.impl.EmbeddableTMHelper#5748c911
Can someone help me to interpret this?
EDIT #2
Here's where the complete stack trace of the blocked threads:
LargeThreadPool-thread-148 [217] (BLOCKED)
com.ibm.tx.jta.embeddable.impl.EmbeddableTMHelper.start line: 63
com.ibm.tx.jta.util.TxTMHelper.start line: 461
com.ibm.tx.util.TMHelper.start line: 74
com.ibm.tx.jta.util.TxTMHelper.checkTMState line: 500
com.ibm.tx.util.TMHelper.checkTMState line: 116
com.ibm.tx.jta.impl.TranManagerSet.registerResourceInfo line: 270
com.ibm.ws.transaction.services.TransactionManagerService.registerResourceInfo line: 260
com.ibm.ejs.j2c.ConnectionManager.registerXAResourceInfo line: 2537
com.ibm.ejs.j2c.ConnectionManager.<init> line: 509
com.ibm.ejs.j2c.ConnectionManagerServiceImpl.getConnectionManager line: 407
com.ibm.ejs.j2c.ConnectionManagerServiceImpl.getConnectionManager line: 54
com.ibm.ws.jca.cm.AbstractConnectionFactoryService.createResource line: 146
com.ibm.ws.injectionengine.osgi.internal.IndirectJndiLookupObjectFactory.createResourceWithFilter line: 346
com.ibm.ws.injectionengine.osgi.internal.IndirectJndiLookupObjectFactory.createResource line: 319
com.ibm.ws.injectionengine.osgi.internal.IndirectJndiLookupObjectFactory.getObjectInstance line: 133
com.ibm.ws.injectionengine.osgi.internal.IndirectJndiLookupObjectFactory.getObjectInstance line: 99
com.ibm.wsspi.injectionengine.InjectionBinding.getInjectionObjectInstance line: 1556
com.ibm.wsspi.injectionengine.InjectionBinding.getInjectionObject line: 1433
com.ibm.wsspi.injectionengine.InjectionBinding.getInjectionObject line: 1389
com.ibm.ws.injectionengine.osgi.internal.naming.InjectionJavaColonHelper.getObjectInstance line: 116
com.ibm.ws.jndi.url.contexts.javacolon.internal.JavaURLContext.lookup line: 333
com.ibm.ws.jndi.url.contexts.javacolon.internal.JavaURLContext.lookup line: 371
org.apache.aries.jndi.DelegateContext.lookup line: 161
javax.naming.InitialContext.lookup line: 417
pt.sibs.epms.persistence.utils.EntityManagerFactoryController.jndiLookupUsed line: 264
pt.sibs.epms.persistence.utils.EntityManagerFactoryController.checkConfiguration line: 115
pt.sibs.epms.persistence.utils.EntityManagerFactoryController.<init> line: 95
pt.sibs.epms.persistence.utils.EntityManagerFactoryController.<init> line: 51
pt.sibs.epms.persistence.utils.EntityManagerFactoryController$SingletonHolder.<clinit> line: 81
pt.sibs.epms.persistence.utils.EntityManagerFactoryController.getInstance line: 88
pt.sibs.epms.util.logging.LoggerConfiguration.<clinit> line: 33
pt.sibs.epms.ecc.renderer.HtmlFormRenderer.<clinit> line: 25
java.lang.Class.forName0 line: not available [native method]
java.lang.Class.forName line: 348
com.ibm.ws.webcontainer.osgi.webapp.WebApp.addClassToHandlesTypesStartupSet line: 1104
com.ibm.ws.webcontainer.osgi.webapp.WebApp.scanForHandlesTypesClasses line: 1038
com.ibm.ws.webcontainer.webapp.WebApp.initializeServletContainerInitializers line: 2493
com.ibm.ws.webcontainer.webapp.WebApp.initialize line: 1037
com.ibm.ws.webcontainer.webapp.WebApp.initialize line: 6545
com.ibm.ws.webcontainer.osgi.DynamicVirtualHost.startWebApp line: 466
com.ibm.ws.webcontainer.osgi.DynamicVirtualHost.createRunnableHandler line: 264
com.ibm.ws.webcontainer.osgi.DynamicVirtualHost.createRunnableHandler line: 329
com.ibm.ws.http.internal.VirtualHostImpl.discriminate line: 251
com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.ready line: 301
com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleDiscrimination line: 471
com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleNewRequest line: 405
com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.processRequest line: 285
com.ibm.ws.http.channel.internal.inbound.HttpICLReadCallback.complete line: 66
com.ibm.ws.channel.ssl.internal.SSLReadServiceContext$SSLReadCompletedCallback.complete line: 1777
com.ibm.ws.tcpchannel.internal.WorkQueueManager.requestComplete line: 504
com.ibm.ws.tcpchannel.internal.WorkQueueManager.attemptIO line: 574
com.ibm.ws.tcpchannel.internal.WorkQueueManager.workerRun line: 929
com.ibm.ws.tcpchannel.internal.WorkQueueManager$Worker.run line: 1018
java.util.concurrent.ThreadPoolExecutor.runWorker line: 1142
java.util.concurrent.ThreadPoolExecutor$Worker.run line: 617
java.lang.Thread.run line: 745
And the second thread:
LargeThreadPool-thread-3 [33] (BLOCKED)
com.ibm.tx.jta.embeddable.impl.EmbeddableTMHelper.start line: 63
com.ibm.tx.jta.util.TxTMHelper.start line: 461
com.ibm.tx.util.TMHelper.start line: 74
com.ibm.tx.jta.util.TxTMHelper.checkTMState line: 500
com.ibm.tx.util.TMHelper.checkTMState line: 116
com.ibm.tx.jta.impl.TranManagerSet.begin line: 167
com.ibm.ejs.csi.TranStrategy.beginGlobalTx line: 593
com.ibm.ejs.csi.Required.preInvoke line: 56
com.ibm.ejs.csi.TransactionControlImpl.preInvoke line: 222
com.ibm.ejs.container.EJSContainer.preInvokeActivate line: 3176
com.ibm.ejs.container.EJSContainer.EjbPreInvoke line: 2576
com.ibm.ejs.container.TimedObjectWrapper.invokeCallback line: 84
com.ibm.ejs.container.TimerNpRunnable.doWork line: 196
com.ibm.ejs.container.TimerNpRunnable.run line: 103
java.util.concurrent.Executors$RunnableAdapter.call line: 511
java.util.concurrent.FutureTask.run line: 266
java.util.concurrent.ThreadPoolExecutor.runWorker line: 1142
java.util.concurrent.ThreadPoolExecutor$Worker.run line: 617
java.lang.Thread.run line: 745
Edit #3
The thread that is WAITING and, apparently, blocking the other two threads:
LargeThreadPool-thread-38 [103] (WAITING)
java.lang.Object.wait line: not available [native method]
java.lang.Object.wait line: 502
com.ibm.tx.jta.impl.EventSemaphore.waitEvent line: 71
com.ibm.tx.jta.impl.RecoveryManager.waitForReplayCompletion line: 1273
com.ibm.tx.jta.impl.TxRecoveryAgentImpl.initiateRecovery line: 413
com.ibm.ws.recoverylog.spi.RecoveryDirectorImpl.directInitialization line: 751
com.ibm.ws.recoverylog.spi.RecoveryDirectorImpl.driveLocalRecovery line: 1240
com.ibm.ws.recoverylog.spi.RecLogServiceImpl.start line: 125
com.ibm.tx.jta.embeddable.impl.EmbeddableTMHelper.start line: 130
com.ibm.tx.jta.util.TxTMHelper.start line: 461
com.ibm.tx.util.TMHelper.start line: 74
com.ibm.tx.jta.util.TxTMHelper.checkTMState line: 500
com.ibm.tx.util.TMHelper.checkTMState line: 116
com.ibm.tx.jta.impl.TranManagerSet.begin line: 167
com.ibm.ws.transaction.services.TransactionManagerService.begin line: 281
com.ibm.ws.concurrent.persistent.internal.PersistentExecutorImpl$PollingTask.run line: 2239
Not sure if it is related, but ffdc is showing the following exception:
------Start of DE processing------ = [09-11-2016 14:41:09:006 GMT]
Exception = com.ibm.ws.recoverylog.spi.LogIncompatibleException
Source = com.ibm.ws.recoverylog.spi.LogHandle
probeid = 326
Stack Dump = com.ibm.ws.recoverylog.spi.LogIncompatibleException
at com.ibm.ws.recoverylog.spi.LogFileHandle.fileOpen(LogFileHandle.java:522)
at com.ibm.ws.recoverylog.spi.LogHandle.openLog(LogHandle.java:324)
at com.ibm.ws.recoverylog.spi.MultiScopeRecoveryLog.openLog(MultiScopeRecoveryLog.java:602)at com.ibm.ws.recoverylog.spi.RecoveryLogImpl.openLog(RecoveryLogImpl.java:77)
at com.ibm.tx.jta.impl.RecoveryManager.run(RecoveryManager.java:1835)
at java.lang.Thread.run(Thread.java:745)
In your Edit #3, the thread that is waiting in
com.ibm.tx.jta.impl.RecoveryManager.waitForReplayCompletion
will have spawned another recoveryManager thread who's role is to access the transaction log files in your flesystem. That other thread Should do the minimal amount of file processing necessary before signalling to the waiting thread that it may continue. Can you see another thread with a stack containing
com.ibm.tx.jta.impl.RecoveryManager.run ?
I am concerned about the LogIncompatibleException. It suggests that the transaction log files on your filesystem are
corrupt. This should not cause the server to hang and I believe you've hit a product defect.
If you need to make progress quickly, it may be appropriate in your scenario to delete the transaction log files.
Please note that this is something we only suggest to customers with extreme care as the transaction logs ensure
the integrity of distributed transactions. In a production environment we'd generally recommend that such action
is only taken under the guidance of IBM Level 3 Service. But in a test/evaluation scenario it can be applicable.
The Liberty transaction log info is stored in the /wlp/usr/servers//tranlog
directory. If appropriate the tranlog and partnerlog subdirectories may be deleted and the server restarted.