We have the context broker configured with cygnus, and the problem that we are getting, is quite strange.
It happen whhen I inserted a new type of element in Context Broker, and we have a subscription to send that element to cosmos. Cygnus send it properly, and create the corresponding hdfs file in cosmos, but when cosmos try to create the Hive table occurs an error.
That is what the cygnus log says.
05 oct 2015 12:51:43,777 INFO [SinkRunner-PollingRunner-DefaultSinkProcessor]
(com.telefonica.iot.cygnus.backends.hdfs.HDFSBackendImpl.provisionHiveTable:225)
- Creating Hive external table=[table_name]
After some hours, we get the following message:
05 oct 2015 18:00:02,489 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor]
(com.telefonica.iot.cygnus.backends.hive.HiveBackend.doCreateTable:77)
- Runtime error (The Hive table cannot be created.
Hive query='create external table [table_name] (columns) row format serde
'org.openx.data.jsonserde.JsonSerDe' location '[hdfs_folder_location]''.
Details=org.apache.thrift.transport.TTransportException)
05 oct 2015 18:00:02,490 WARN [SinkRunner-PollingRunner-DefaultSinkProcessor]
(com.telefonica.iot.cygnus.backends.hdfs.HDFSBackendImpl.provisionHiveTable:237)
- The HiveQL external table could not be created, but Cygnus can continue working... Check your Hive/Shark installation
Between those two events all the actions that should be added in hdfs, they won't be added until we get the error.
Cygnus Version is: 0.8.2
We are using the open cosmos.
This is a well known issue in Cygnus 0.8.2, here you can find a workaround. Nevertheless, this has been fixed in Cygnus 0.9.0 (in order to avoid you have to install it from the sources, I'll upload along the day a RPM to the FIWARE repo).
Related
I am just finding wazuh and wanted to test it out. I have tried the step by step and the scripted. Cannot get it to work. So I am consolidating my deployment to just 1 VM and am trying to run the quickstart to get going. When I try to install I get the below error. I cannot find anything related to wazuh-indexer 4.3.1-1. Thoughts?
10/05/2022 12:52:37 INFO: Created wazuh-install-files.tar. It contains the Wazuh cluster key, certificates, and passwords necessary for installation.
10/05/2022 12:52:37 INFO: --- Wazuh indexer ---
10/05/2022 12:52:37 INFO: Starting Wazuh indexer installation.
Updating Subscription Management repositories.
EL-8 - Wazuh 75 kB/s | 3.4 kB 00:00
No match for argument: wazuh-indexer-4.3.1-1
Error: Unable to find a match: wazuh-indexer-4.3.1-1
10/05/2022 12:52:48 ERROR: Wazuh indexer installation failed.
10/05/2022 12:52:48 INFO: --- Removing existing Wazuh installation ---
10/05/2022 12:52:48 INFO: Installation cleaned. Check the /var/log/wazuh-install.log file to learn more about the issue.
The root of that error is that the script is trying to download
wazuh-indexer-4.3.1-1 instead of the current wazuh-indexer-4.3.0-1.
It seems that the production script included this bug for a short period of time. Please download the script and install it again.
Sorry for the inconvenience.
You are correct. After waiting a few days, the bug was fixed. But am now hitting a new bug. When trying to run filebeat setup --dashboards. I get an error: Exiting: Kibana API is not available in Kibana version 1.2.0
I have the setting:
compatibility.override_main_response_version: true
on my opensearch.yml but still no avail.
If you are using the wazuh-install script, it is not required to perform any further configuration.
In order to troubleshoot this issue, could you please provide us with the following information:
Architecture: Single host or multi-node
Installation method: Step by step installation
Documentation page you follow to install your environment
OS of your nodes
I recommend you to join Wazuh slack channel (https://wazuh.com/community/) and move this issue there, or into the Wazuh google group in order to be able to correctly monitor this problem
My database is in another Linux node lets suppose xxx.xx.xxx.xxx:3306 and I am running the standalone command of kafka connect from my Windows machine. My server.properties code is
name=test-source-mysql-jdbc-autoincrement
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
tasks.max=1
connection.url=jdbc:mysql://xxx.xx.xxx.xxx:3306/databasename?user=username&useSSL=false
&password=password
mode=incrementing
incrementing.column.name=columnname
topic.prefix=mysqlname-databasename-database-
I am Receiving the following error
Mar 28, 2019 11:55:41 AM org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
WARNING: A provider org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. D
ue to constraint configuration problems the provider org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource will be ignored.
Mar 28, 2019 11:55:41 AM org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
WARNING: A provider org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to
constraint configuration problems the provider org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource will be ignored.
Mar 28, 2019 11:55:41 AM org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
WARNING: A provider org.apache.kafka.connect.runtime.rest.resources.RootResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constr
aint configuration problems the provider org.apache.kafka.connect.runtime.rest.resources.RootResource will be ignored.
Mar 28, 2019 11:55:41 AM org.glassfish.jersey.internal.Errors logErrors
WARNING: The following warnings have been detected: WARNING: The (sub)resource method createConnector in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotati
on.
WARNING: The (sub)resource method listConnectors in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation.
WARNING: The (sub)resource method listConnectorPlugins in org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource contains empty path annotation.
WARNING: The (sub)resource method serverInfo in org.apache.kafka.connect.runtime.rest.resources.RootResource contains empty path annotation.
[2019-03-28 11:55:42,108] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone)
java.lang.IllegalArgumentException: Number of groups must be positive.
at org.apache.kafka.connect.util.ConnectorUtils.groupPartitions(ConnectorUtils.java:41)
at io.confluent.connect.jdbc.JdbcSourceConnector.taskConfigs(JdbcSourceConnector.java:148)
at org.apache.kafka.connect.runtime.Worker.connectorTaskConfigs(Worker.java:305)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.recomputeTaskConfigs(StandaloneHerder.java:307)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.updateConnectorTasks(StandaloneHerder.java:333)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:211)
at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:107)
Any help is appreciated.
The error Number of groups must be positive. indicates that Kafka Connect JDBC source has not found any tables/objects to fetch from the database. Looking at your config, you've not specified table.whitelist / table.blacklist so that could be why. Another reason would be if the user is not authorised to select from any of the available objects. This article goes into more detail.
I'm new to Redhat OpenShift and trying to deploy node application (with angularjs + mysql) and running into build issues.
Using openshift console created node application and in advanced options pointed to the private repository and linked configured secret (ssh key to private repository).
My build is failing with "Assemble script failed". Pasting the logs as below (from console - obfuscated private keys and values).
Not sure if I'm missing some configurations. Appreciate help on this.
Cloning "ssh://username#bitbucket.org/username/my-app.git" ...
Commit: xxxxxxxxxxxxxxxxx (Fixed readme)
Author: Name <email>
Date: Wed Sep 6 19:50:59 2017 -0700
Pulling image "docker-
registry.default.svc:5000/openshift/nodejs#sha256:0000000000000" ...
---> Installing application source
---> Building your Node application from source
Current git config
url.https://github.com.insteadof=git#github.com:
url.https://.insteadof=ssh://
url.https://github.com.insteadof=ssh://git#github.com
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=ssh://username#bitbucket.org/username/my-app.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master
---> Installing dependencies
---> Using 'npm install -s --only=production'
error: build error: non-zero (13) exit code from docker-
registry.default.svc:5000/openshift/nodejs#sha256:0000000000000
Pls note that my source code is hosted on private repository and per log above it appears openshift is able to access the repository and download the source code.
Thanks Graham for the pointer. I recreated the application and this time in advanced options on the web console selected 1 GB (from default of 500 mb) after which my build worked fine.
I ran the following batch file on our SQL Server "9". It is an SSIS 2012 package with no connections, no tasks, no code. It's an empty package.
ECHO before dtexe!
REM "DTExec.exe" /FILE "E:\SSIS\MyAppName\Packages\Empty2012Package.dtsx"
"D:\SQL2012\110\DTS\Binn\DTExec.exe" /FILE "E:\SSIS\MyAppName\Packages\Empty2012Package.dtsx"
SET RESULT=%errorlevel%
ECHO RESULT AFTER DTEXEC=%RESULT%
exit /B %RESULT%
Here's my output:
----------------------------------------------------------------
Output of messages for workload object TEST/GHG1990R.28/MAIN
Start date Tue Oct 06 20:45:38 2015
----------------------------------------------------------------
C:\Users\MyServerId>ECHO before dtexe!
before dtexe!
C:\Users\MyServerId>REM "DTExec.exe" /FILE "E:\SSIS\MyAppName\Packages\Empty2012Package.dtsx"
C:\Users\MyServerId>"D:\SQL2012\110\DTS\Binn\DTExec.exe" /FILE "E:\SSIS\MyAppName\Packages\Empty2012Package.dtsx"
Microsoft (R) SQL Server Execute Package Utility
Version 11.0.5058.0 for 64-bit
Copyright (C) Microsoft Corporation. All rights reserved.
Started: 8:45:39 PM
Could not load package "E:\SSIS\MyAppName\Packages\Empty2012Package.dtsx" because of error 0x80131534.
Description: The package failed to load due to error 0x80131534 "(null)". This occurs when CPackage::LoadFromXML fails.
Source: {BA581388-EFF5-4BC7-89E7-48E11D6DF0AE}
Started: 8:45:39 PM
Finished: 8:45:39 PM
Elapsed: 0.094 seconds
C:\Users\MyServerId>SET RESULT=5
C:\Users\MyServerId>ECHO RESULT AFTER DTEXEC=5
RESULT AFTER DTEXEC=5
C:\Users\MyServerId>exit /B 5
I get the same error regardless of whether I run with the 32 or 64 bit DTEXEC utility. When I run the same batch file against the same package on another SQL Server, server "80", I get no error.
All of the packages work on server 80, none work on 9. I get the above error every time on server 9.
When I execute this on both 80 and 9,
select ##version
I get the same result
Microsoft SQL Server 2012 - 11.0.5058.0 (X64)
May 14 2014 18:34:29
Copyright (c) Microsoft Corporation
Enterprise Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1) (Hypervisor)
I am able to successfully use the sqlcmd and bcp utilities on 9, but not the dtexec utility.
The error that I am getting seems to suggest that I am using an old version of dtexec to open up a newer package, for example, trying to use a SQL 2008 dtexec util to execute a new SQL 2012 package.
Clearly the packages are not corrupt because all of the packages work on 80 and copies of the exct files on 9 all do not work.
The only thing I can think of is a installation issue with SQL Server, for example, a corrupt install or unregistered dll.
As far as I can tell, this is the first and only version of SQL Server installed on these machines, so I shoould not be acidentily runnning an old version of DTEXEC with a newer 2012 package.
Here is the complete Empty2012Package.dtsx file:
<?xml version="1.0"?>
<DTS:Executable
DTS:refId="Package" xmlns:DTS="www.microsoft.com/SqlServer/Dts"
DTS:ExecutableType="SSIS.Package.3"
DTS:CreatorName="MyDOMAIN\MyLanId"
DTS:CreatorComputerName="MYCPTR"
DTS:CreationDate="7/2/2015 8:26:52 AM"
DTS:PackageType="5"
DTS:VersionBuild="2"
DTS:VersionGUID="{00FD1F37-6375-4942-91D3-9ECA0B131FFD}"
DTS:LastModifiedProductVersion="11.0.2100.60"
DTS:LocaleID="1033"
DTS:ObjectName="Empty2012Package"
DTS:DTSID="{1539BA56-F17B-438E-A874-A2151F2F79C5}"
DTS:CreationName="SSIS.Package.3">
<DTS:Property
DTS:Name="PackageFormatVersion">6</DTS:Property>
<DTS:Variables />
<DTS:Executables />
<DTS:DesignTimeProperties><![CDATA[<?xml version="1.0"?>
<!--This CDATA section contains the layout information of the package. The section includes information such as (x,y) coordinates, width, and height.-->
<!--If you manually edit this section and make a mistake, you can delete it. -->
<!--The package will still be able to load normally but the previous layout information will be lost and the designer will automatically re-arrange the elements on the design surface.-->
<Objects
Version="sql11">
<!--Each node below will contain properties that do not affect runtime behavior.-->
</Objects>]]></DTS:DesignTimeProperties>
</DTS:Executable>
Since I used SQL 2012 to create the dtsx file, I knew that is was a 2012 package. However, according to this web page,
http://www.techbrothersit.com/2014/09/ssis-how-to-find-version-of-ssis.html
PackageFormatVersion of 6 indicates a 2012 package.
I don't have access to the server to see the versions of each dtexec.exe file. I hope to get this information soon.
When I run the same batch file with the same package on 80, as mentioned above, it works:
----------------------------------------------------------------
Output of messages for workload object C1_GHG1_TEST/GHG1999I.11/MAIN
Start date Tue Oct 06 22:21:08 2015
----------------------------------------------------------------
C:\Users\MyServerId>ECHO before dtexe!
before dtexe!
C:\Users\MyServerId>REM "DTExec.exe" /FILE "E:\SSIS\MyAppName\Packages\Empty2012Package.dtsx"
C:\Users\MyServerId>"D:\SQL2012\110\DTS\Binn\DTExec.exe" /FILE "E:\SSIS\MyAppName\Packages\Empty2012Package.dtsx"
Microsoft (R) SQL Server Execute Package Utility
Version 11.0.5058.0 for 64-bit
Copyright (C) Microsoft Corporation. All rights reserved.
Started: 10:21:08 PM
DTExec: The package execution returned DTSER_SUCCESS (0).
Started: 10:21:08 PM
Finished: 10:21:08 PM
Elapsed: 0.203 seconds
C:\Users\MyServerId>SET RESULT=0
C:\Users\MyServerId>ECHO RESULT AFTER DTEXEC=0
RESULT AFTER DTEXEC=0
C:\Users\MyServerId>exit /B 0
Any ideas on what may be causing this error? I'm thinking that this is a server SQL Server setup issue rather than something that I can control.
Your thoughts?
Update
The problem is related to security permissions. When we try running the job in the CA Scheduler udner MyServiceID, an Admin with access to the box then checked the Event Log and found this error:
The user has not been granted the requested logon type at this machine.
Furthermore, When a user who is in the local Admin group remotes to this server and launches the same batch file, the job runs successfully.
According to this link,
https://technet.microsoft.com/en-us/library/cc732593(v=ws.10).aspx
The error can be resoled by granting the right to "Allow User To Log On Locally." The user is already granted "Allow Log On As Batch" rights. When I dop a "net user MyServiceId" command from the command line, I also see that the ID is active and allowed on all Workstations at all times.
When I requested "Allow user to LogOn Locally" rights to be granted, apparently even the "Admins" could not grant this right because is is disallowed by Group Policy. Because this is the only server where this is happening, I am left wondering what is different about this server assuming that all serevrs share the same Group Policy. One thing that we noticed is that the OWNER property on the dtsx file is not MyServiceId and in environments. I don't have the righs to change this but I don't thing that this is it. The MyServiceAccount does have full control of the folder where the batch files and packages reside.
Update 2
Since this is now clearly a security issue, perhaps it would help if I try an state the question after stating the known facts:
MyServiceId has execute access to the local folder, E:\SSIS\MyAppName\Packages\, in which SSIS package resides
MyServiceId has execute access to the local folder, D:\SQL2012\110\DTS\Binn\, in which DTEXEC resides
MyServiceId has LogOn As Batch rights on the Server.
MyServiceId is an active account that is allowed on all Servers and Workstations
Question:
Given the simple one line batch file that calls DTEXEC passing a package path as a param, what could be contributing to the error that I am getting? What additional things can I look for? Sicne "Allow User To Log On locally" is never set on other server, what can be different on this server that is contributing to this error:
The user has not been granted the requested logon type at this machine.
Is the value of the OWNER property of the dtsx file relevant?
Update 3
Calling process Name: C:\Windows\System32\LSASS.exe
Is this a hint?
Update 4
In production, where it works,the service id is executing with a Network Logon type of 3, which is interactive. On 9, where it's giving us this error, it is executing with a logon type of 2, which is interactive. (We do not have the option to turn on "Allow Log On Locally" for the uyser on 9. This is verboten.
What would account for the same ID running on 2 different machines on teh same domain, to run as different user types?
I encountered this error shortly after making a modification to my machine.config file on my workstation. I resolved my issues in the machine.config (I had placed a node under the wrong parent node) and this error went away.
Unfortunately, since the error is vague, this may not be the solution for your situation. Just thought I'd share in case it helps someone.
I changed my SQL service account to local system. My SSIS services were also running from local system only.
So, ideally we can try keeping both as same services account.
NOTE: this needs to be applied only in case we are not able to change the access rights of SQL agent service account.
Am experienced with Django, Apache, WSGI, MySql, etc. and have this environment setup and working fine on another OS X computer running Lion. Django session backend is db, session middleware and app are both enabled properly in settings. During first request of site view, we set a request.session key/value, which this first time thru works fine. On subsequent view though when we check if the key/value exists, we get a server-level 500 error that even with Debug mode on doesn't make it to the python interpreter to generate a full stack trace exception. The apache log generates the following messages...
[Mon Apr 16 14:26:22 2012] [notice] Apache/2.2.21 (Unix) DAV/2 mod_wsgi/3.3 Python/2.7.1 mod_ssl/2.2.21 OpenSSL/0.9.8r configured -- resuming normal operations
[Mon Apr 16 14:26:27 2012] [info] mod_wsgi (pid=2362): Create interpreter 'snap.joe|'.
[Mon Apr 16 14:26:27 2012] [info] [client 127.0.0.1] mod_wsgi (pid=2362, process='snap', application='snap.joe|'): Loading WSGI script '/var/www/venvs/snap_env/snap/wsgi/wsgi.py'.
[Mon Apr 16 14:26:32 2012] [error] [client 127.0.0.1] Premature end of script headers: wsgi.py
[Mon Apr 16 14:26:33 2012] [notice] child pid 2362 exit signal Bus error (10)
Have checked that a MySQL db table django_session row is added properly with session_data and also the cookie set in the browser after the first request contains the proper/matching session_id. If I add a simple request.session.flush() just before the existing session code, we can bypass the error obviously because there is never anything in the session. One more thing, the value we are adding to the session key/value store is a Django QuerySet object. Again, this is working on a live CentOS server, and also another Mac OS X Lion machine (albeit running Python 2.6 instead of 2.7).
Any ideas folks? Many thanks!
See documented reasons at:
http://code.google.com/p/modwsgi/wiki/FrequentlyAskedQuestions#Apache_Process_Crashes
and follow further links for possible workarounds.
In short, can be shared library version conflict, using mod_python at same time, or extension module for Python that doesn't work with sub interpreter.
If this is only Python site on that Apache, set:
WSGIApplicationGroup %{GLOBAL}
for one potential quick solution.
A few thoughts:
1) are you running the same versions of modules? Django just recently updated (1.4), so if you did a pip install and didn't specify a version your production might have an older version while your development has the newer one.
2) Can you test this using just the manage.py runserver? My suspicions are that it's something to do with the apache config (so if it works fine in runserver, that means the issue is with apache).