Do you know how can I turn Warnings, Notes, Errors in HtmlUnit off?
Put this somewhere around the start of your code; it will shut its dirty mouth:
LogFactory.getFactory().setAttribute("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("org.apache.commons.httpclient").setLevel(Level.OFF);
webClient = new WebClient(bv);
webClient.setCssEnabled(false);
webClient.setIncorrectnessListener(new IncorrectnessListener() {
#Override
public void notify(String arg0, Object arg1) {
// TODO Auto-generated method stub
}
});
webClient.setCssErrorHandler(new ErrorHandler() {
#Override
public void warning(CSSParseException exception) throws CSSException {
// TODO Auto-generated method stub
}
#Override
public void fatalError(CSSParseException exception) throws CSSException {
// TODO Auto-generated method stub
}
#Override
public void error(CSSParseException exception) throws CSSException {
// TODO Auto-generated method stub
}
});
webClient.setJavaScriptErrorListener(new JavaScriptErrorListener() {
#Override
public void timeoutError(HtmlPage arg0, long arg1, long arg2) {
// TODO Auto-generated method stub
}
#Override
public void scriptException(HtmlPage arg0, ScriptException arg1) {
// TODO Auto-generated method stub
}
#Override
public void malformedScriptURL(HtmlPage arg0, String arg1, MalformedURLException arg2) {
// TODO Auto-generated method stub
}
#Override
public void loadScriptError(HtmlPage arg0, URL arg1, Exception arg2) {
// TODO Auto-generated method stub
}
});
webClient.setHTMLParserListener(new HTMLParserListener() {
#Override
public void warning(String arg0, URL arg1, int arg2, int arg3, String arg4) {
// TODO Auto-generated method stub
}
#Override
public void error(String arg0, URL arg1, int arg2, int arg3, String arg4) {
// TODO Auto-generated method stub
}
});
webClient.setThrowExceptionOnFailingStatusCode(false);
webClient.setThrowExceptionOnScriptError(false);
The code in Arsen Zahray's answer helped in removing almost all the logs generated by HtmlUnit.
But one edit helps to remove them all. Use:
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);
instead of:
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit").setLevel(Level.OFF);
To remove all output from the latest version of HtmlUnit you just have to add these lines in a static block or in your main class:
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);
System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");
It is NOT needed to override any method as some other answers state.
Try the following code to turn the logging level down to off:
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);
Here you can get info on how to manipulate logging of HtmlUnit.
This is what I added to my log4j.properties in order to disable verbose debugging messages from HtmlUnit components:
# Set specific logger levels.
log4j.logger.org.mortbay.log=fatal
log4j.logger.org.apache.http=fatal
log4j.logger.org.apache.http.headers=fatal
log4j.logger.org.apache.http.wire=fatal
# For HttpClient 3, which is used by FirefoxDriver
log4j.logger.httpclient.wire=fatal
log4j.logger.org.apache.commons=fatal
log4j.logger.com.gargoylesoftware.htmlunit=fatal
log4j.logger.com.gargoylesoftware.htmlunit.WebTestCase=fatal
# Change this to TRACE when enabling the debugger.
log4j.logger.com.gargoylesoftware.htmlunit.javascript.DebugFrameImpl=fatal
I am using the code below and it works perfectly:
LogFactory.getFactory().setAttribute("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("org.apache.commons.httpclient").setLevel(Level.OFF);
Just add this string to your log4.properties:
log4j.logger.com.gargoylesoftware.htmlunit=fatal
Turn the loggers off. But that is not a good solution, since you might want to have some uncommon issues in the logs.
I know HtmlUnit produces a lot of unimportant exceptions, warnings, etc. You can suppress of few of those using:
client.getOptions().setThrowExceptionOnFailingStatusCode(false);
client.getOptions().setThrowExceptionOnScriptError(false);
client.getOptions().setPrintContentOnFailingStatusCode(false);
Now in HtmlUnit 2.9, WebClient.setCssErrorHandler(new SilentCssErrorHandler()) can conveniently ignore the warnings and errors. For example:
#Override
protected WebClient modifyWebClient(WebClient client) {
// currently does nothing, but may be changed in future versions
WebClient modifiedClient = super.modifyWebClient(client);
modifiedClient.getOptions().setThrowExceptionOnScriptError(false);
modifiedClient.setCssErrorHandler(new SilentCssErrorHandler());
return modifiedClient;
}
Have a look at the docs.
There is a sample log4 file used by the test suite, you can find it here, you can disable everything if you wish.
This worked for me:
#Test
public void homePage() throws Exception {
final WebClient webClient = new WebClient();
webClient.setThrowExceptionOnScriptError(false);
final HtmlPage page = webClient.getPage("http://localhost:8080/web/guest/home");
Try adding this to your code:
LogFactory.getFactory().setAttribute("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");
Basically, this makes the logger log to NoOpLog, which doesn't write the log information anywhere.
I must be doing something different to everyone above. I have htmlunit set up as a Spring project currently and removing the logs required adding a logback.xml to my resources dir. Add the following as logback.xml to your main/java/resources dir - this will only output INFO level log statements and nothing below (When to use the different log levels)
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<!--<include resource="org/springframework/boot/logging/logback/base.xml"/>-->
<!--<logger name="org.springframework.web" level="INFO"/>-->
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%d{yyyy-MM-dd HH:mm:ss} [%thread] [%logger{20}] %-5level - %msg%n</pattern>
</encoder>
</appender>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>gdaxDesktop.log</file>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss} [%thread] %logger{20} %-5level - %msg%n</pattern>
</encoder>
</appender>
<root level="info">
<!--<appender-ref ref="FILE"/>-->
<appender-ref ref="CONSOLE"/>
</root>
</configuration>
One option which worked well for me is to change the HtmlUnit logger to log to a different file just so that I have those errors in case I need to refer it some time and it also doesn't clutter up my main logs.
Below is the log4j change I made to log4j.properties:
log4j.logger.com.gargoylesoftware.htmlunit=ERROR, HTMLUNIT
log4j.additivity.com.gargoylesoftware.htmlunit=false
log4j.appender.HTMLUNIT = org.apache.log4j.RollingFileAppender
log4j.appender.HTMLUNIT.layout=org.apache.log4j.PatternLayout
log4j.appender.HTMLUNIT.layout.ConversionPattern=%m%n
log4j.appender.HTMLUNIT.File=logs/HtmlUnitLog4j.log
log4j.appender.HTMLUNIT.MaxFileSize=5MB
log4j.appender.HTMLUNIT.MaxBackupIndex=5
If you don't need JavaScript support, it is the easiest way to disable it:
WebClient client = new WebClient(BrowserVersion.BEST_SUPPORTED);
client.getOptions().setThrowExceptionOnFailingStatusCode(false);
client.getOptions().setPrintContentOnFailingStatusCode(false);
client.getOptions().setThrowExceptionOnScriptError(false);
client.getOptions().setJavaScriptEnabled(false);
client.setCssErrorHandler(new SilentCssErrorHandler());
client.setHTMLParserListener(new HTMLParserListener() {
#Override
public void error(String message, URL url, String html, int line, int column, String key) {
}
#Override
public void warning(String message, URL url, String html, int line, int column, String key) {
}
});
You also disable exceptions and log on failing status codes, JavaScript, CSS errors and HTML parse errors.
If you need JavaScript support you can use a custom implementation for JavaScript errors:
client.setJavaScriptErrorListener(new JavaScriptErrorListener() {
#Override
public void timeoutError(HtmlPage arg0, long arg1, long arg2) {
}
#Override
public void scriptException(HtmlPage arg0, ScriptException arg1) {
}
#Override
public void malformedScriptURL(HtmlPage arg0, String arg1, MalformedURLException arg2) {
}
#Override
public void loadScriptError(HtmlPage arg0, URL arg1, Exception arg2) {
}
});
If you don't need you also can just disable it:
client.getOptions().setCssEnabled(false);
So there is no need to configure any other Logger.
I am using spring-boot, only solved with this:
import com.gargoylesoftware.htmlunit.SilentCssErrorHandler;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.javascript.SilentJavaScriptErrorListener;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
#Configuration
public class HtmlUnitConfiguration {
#Bean
public WebClient webClient() {
WebClient webClient = new WebClient();
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.setJavaScriptErrorListener(new SilentJavaScriptErrorListener());
webClient.setCssErrorHandler(new SilentCssErrorHandler());
return webClient;
}
}
And then calling the bean with #Autowired or in class constructor. And without this line:
webClient.getOptions().setThrowExceptionOnScriptError(false);
The two lines under it will throw a bizarre error. This line has the magic.
I'm adding this answer for logback users:
<!-- LOG "com.gargoylesoftware.htmlunit.javascript*" at OFF (closed) level -->
<logger name="com.gargoylesoftware.htmlunit.javascript"
level="OFF" additivity="false">
<appender-ref ref="RollingFile" />
<appender-ref ref="Console" />
</logger>
Setting the logger level off as previous answers from apache logging and java util logging did not work for me (at least when im running tests on my local computer using htmlunit).
And passing the dummy listener as javascriptErrorListener for WebClient instance also was not working for me.
So it was the only solution for closing the annoying javascript exception logs.
Try this, it worked for me.
WebClient webClient = new WebClient();
webClient.getOptions().setJavaScriptEnabled(false);
webClient.getOptions().setCssEnabled(false);
Related
When using apache htttpasyncclient how does one handle various exception cases properly? Consider the following pseudo code based on this example where I have added the consumer parameter to the execute call. The intention is to make an asynchronous http request where the data is processed as a stream when the bytes come in rather than waiting for the full response to be done before processing. Various issues can occur such as a timeout exception on the http request, failure to connect (could be no network), etc. Is it always guaranteed that for example on a timeout with a response not coming back in time that releaseResources() is always called. The question is where does latch.countDown() need to be put in the below code to always guarantee that the await call doesn't just hang no matter what the exception. Is the call to latch.countDown() in StreamConsumer.releaseResources() enough to prevent hanging on the await?
public static void main(final String[] args) throws Exception {
client.execute(HttpAsyncMethods.createGet(u), new StreamConsumer(...), new FutureCallback<Boolean>() {
#Override
public void cancelled() {
// Is latch call needed here?
// latch.countDown();
}
#Override
public void completed(Boolean response) {
// Is latch call needed here?
// latch.countDown();
}
#Override
public void failed(Exception e) {
// Is latch call needed here?
// latch.countDown();
}
});
latch.await();
}
static class StreamConsumer extends AsyncByteConsumer<Boolean> {
#Override
protected void onResponseReceived(final HttpResponse response) {
latch.countDown();
}
#Override
protected void onByteReceived(final ByteBuffer buf, final IOControl ioctrl) throws IOException {
}
#Override
protected void releaseResources() {
latch.countDown();
}
}
CloseableHttpAsyncClient#execute method terminates immediately upon submission of a request into the request execution pipeline and returns a Future object representing the future result of the operation.
Therefore the latch in the example is required to make sure that the client does not get shut down immediately after CloseableHttpAsyncClient#execute call.
If one uses CloseableHttpAsyncClient as a singleton with a defined life-cycle (as one should) synchronization of request completion and the client shutdown may be unnecessary.
I have the following code in my performFinish() method of my Wizard Class :
public boolean performFinish() {
try {
getContainer().run(true, false, changeArtifactRunnable());
}
catch (InvocationTargetException | InterruptedException e) {
LoggerClass.logException(e);
}
I want to test Exception for InvocationTargetException and InterruptedException using Mockito.
In the above code, getContainer() method is from org.eclipse.jface.wizard.Wizard class and
public void run(boolean fork, boolean cancelable,
IRunnableWithProgress runnable) throws InvocationTargetException,
InterruptedException;
method is from org.eclipse.jface.operation.IRunnableContext class.
How do I test both the exceptions in performFinish() method?
You can use the expected keyword in order to do so. For example:
#Test(expected = InvocationTargetException.class)
public void testInvocationTargetException() {
\\Invoke the method to be tested under the conditions, such that InvocationTargetException is thrown by it. No need of any assert statements
}
===========================================================================
Edit:
#RunWith(MockitoJUnitRunner.class)
public class EditArtifactWizardTest {
#Spy
//use correct constructor of EditArtifactWizard
private EditArtifactWizard editArtifactWizardSpy=Mockito.spy(new EditArtifactWizard ());
#Test(expected = InvocationTargetException.class)
public void testInvocationTargetException() {
\\Invoke the method to be tested under the conditions, such that InvocationTargetException is thrown by it. No need of any assert statements
Mockito.when(editArtifactWizardSpy.getContainer()).thenThrow(InvocationTargetException.class);
editArtifactWizardSpy.performFinish();
}
}
You can create the Spy of EditArtifactWizard class and mock the behavior of the getContainerMethod.
P.S: Please excuse for typos or compilation error as I am not using any editor.
I'm attempting to serialize my logs to Json. I'm using log4net with a custom layout, but when an exception is logged, I get the following malformed JSON (Note the additional stacktrace info at the end)
Am I missing a setting for log4net, or is this a serialization issue?
UPDATE: This has to be something with log4net, because json.net serializes an Exception perfectly.
***UPDATE (FIXED): Updated code below.
{
"UserSessionId":"4b146c92-fe99-4f78-bbef-720df2cf7473",
"ProcessSessionId":1,
...
"Logger":"testharness.Program","ThreadName":"1",
"ExceptionObject":{
"ClassName":"System.ApplicationException",
"Message":"Test Exception Logging",
"Data":null,
"InnerException":null,
"HelpURL":null,
"StackTraceString":" at testharness.Program.Main(String[] args) in C:\\temp\\testharness\\Program.cs:line 18",
"RemoteStackTraceString":null,
...
"WatsonBuckets":null
},
...
"log4net:HostName":"ol-4RBNMH2"
}}
System.ApplicationException: Test Exception Logging
at testharness.Program.Main(String[] args) in C:\temp\testharness\Program.cs:line 18
log4net configuratiton
<log4net>
<appender name="RollingFileCompositeAppender" type="log4net.Appender.RollingFileAppender">
<file value="c:\\logs\\testharness.txt"/>
<appendToFile value="true"/>
<rollingStyle value="Composite"/>
<datePattern value="yyyy-MM-dd"/>
<maxSizeRollBackups value="-1"/>
<maximumFileSize value="1MB"/>
<countDirection value="1"/>
<preserveLogFileNameExtension value="false"/>
<staticLogFileName value="false"/>
<layout type="Company.log4net.JsonLayout"></layout>
</appender>
<root>
<level value="ALL"/>
<appender-ref ref="RollingFileCompositeAppender"/>
</root>
</log4net>
The Custom Layout class
public class JsonLayout : LayoutSkeleton
{
public JsonLayout() {
IgnoresException = false;
}
...
/// <inheritdoc />
public override void ActivateOptions()
{
}
/// <inheritdoc />
public override void Format(TextWriter writer, LoggingEvent loggingEvent)
{
_customProperties.PhysicalMemory = Process.GetCurrentProcess().WorkingSet64;
var evt = new CustomLoggingEvent(loggingEvent, _customProperties);
writer.Write(JsonConvert.SerializeObject(evt));
}
}
[JsonObject(MemberSerialization.OptIn)]
public class CustomLoggingEvent
{
...
[JsonProperty]
public Exception ExceptionObject { get; set; }
[JsonProperty]
public long PhysicalMemory { get; set; }
[JsonProperty]
public PropertiesDictionary Properties { get; set; }
}
The Test Harness:
internal class Program
{
private static void Main()
{
Console.WriteLine($"{typeof(JsonLayout)}");
var log = log4net.LogManager.GetLogger(typeof(Program));
try
{
log.Debug("hello again world");
throw new ApplicationException("Test Exception Logging");
}
catch (Exception e)
{
log.Error("Exception Thrown", e);
}
Console.ReadLine();
}
}
I downloaded the log4net source and found the issue. When creating a custom layout implementing LayoutSkeleton, if your layout handles the LoggingEvent.ExceptionObject, then the IgnoresException property should be set to false; the default value is true. I updated the code in the question with a constructor wherein the IgnoresException property is set to true. I probably should have waited a day while I researched, but maybe this will help someone else.
I have a Jersey (1.x) based REST service. It uses Jackson 2.4.4 to generate JSON responses. I need to add a newline character at the end of response (cURL users complain that there's no new line in responses). I am using Jersey pretty-print feature (SerializationFeature.INDENT_OUTPUT).
current: {\n "prop" : "value"\n}
wanted: {\n "prop" : "value"\n}\n
I tried using a custom serializer. I need to add \n only at the end of the root object. Serializer is defined per data type, which means, if an instance of such class is nested in a response, I will get \n in the middle of my JSON.
I thought of subclassing com.fasterxml.jackson.core.JsonGenerator.java, overriding close() where i'd add writeRaw('\n'), but that feels very hacky.
Another idea would be to add Servlet filter which would re-write the response from Jersey Filter, adding the \n and incrementing the contentLenght by 1. Seems not only hacky, but also inefficient.
I could also give up Jersey taking care of serializing the content and do ObjectMapper.writeValue() + "\n", but this is quite intrusive to my code (need to change many places).
What is the clean solution for that problem?
I have found these threads for the same problem, but none of them provides solution:
http://markmail.org/message/nj4aqheqobmt4o5c
http://jackson-users.ning.com/forum/topics/add-newline-after-object-serialization-in-jersey
Update
Finally I went for #arachnid's solution with NewlineAddingPrettyPrinter (also bumper Jackson version to 2.6.2). Sadly, it does not work out of the box with Jaskson as JAX-RS Json provider. Changed PrettyPrinter in ObjectMapper does not get propagated to JsonGenerator (see here why). To make it work, I had to add ResponseFilter which adds ObjectWriterModifier (now I can easily toggle between pretty-print and minimal, based on input param ):
#Provider
public class PrettyPrintFilter extends BaseResponseFilter {
public ContainerResponse filter(ContainerRequest request, ContainerResponse response) {
ObjectWriterInjector.set(new PrettyPrintToggler(true));
return response;
}
final class PrettyPrintToggler extends ObjectWriterModifier {
private static final PrettyPrinter NO_PRETTY_PRINT = new MinimalPrettyPrinter();
private final boolean usePrettyPrint;
public PrettyPrintToggler(boolean usePrettyPrint) {
this.usePrettyPrint = usePrettyPrint;
}
#Override
public ObjectWriter modify(EndpointConfigBase<?> endpoint, MultivaluedMap<String, Object> responseHeaders,
Object valueToWrite, ObjectWriter w, JsonGenerator g) throws IOException {
if (usePrettyPrint) g.setPrettyPrinter(new NewlineAddingPrettyPrinter());
else g.setPrettyPrinter(NO_PRETTY_PRINT);
return w;
}
}
}
Actually, wrapping up (not subclassing) JsonGenerator isn't too bad:
public static final class NewlineAddingJsonFactory extends JsonFactory {
#Override
protected JsonGenerator _createGenerator(Writer out, IOContext ctxt) throws IOException {
return new NewlineAddingJsonGenerator(super._createGenerator(out, ctxt));
}
#Override
protected JsonGenerator _createUTF8Generator(OutputStream out, IOContext ctxt) throws IOException {
return new NewlineAddingJsonGenerator(super._createUTF8Generator(out, ctxt));
}
}
public static final class NewlineAddingJsonGenerator extends JsonGenerator {
private final JsonGenerator underlying;
private int depth = 0;
public NewlineAddingJsonGenerator(JsonGenerator underlying) {
this.underlying = underlying;
}
#Override
public void writeStartObject() throws IOException {
underlying.writeStartObject();
++depth;
}
#Override
public void writeEndObject() throws IOException {
underlying.writeEndObject();
if (--depth == 0) {
underlying.writeRaw('\n');
}
}
// ... and delegate all the other methods of JsonGenerator (CGLIB can hide this if you put in some time)
}
#Test
public void append_newline_after_end_of_json() throws Exception {
ObjectWriter writer = new ObjectMapper(new NewlineAddingJsonFactory()).writer();
assertThat(writer.writeValueAsString(ImmutableMap.of()), equalTo("{}\n"));
assertThat(writer.writeValueAsString(ImmutableMap.of("foo", "bar")), equalTo("{\"foo\":\"bar\"}\n"));
}
A servlet filter isn't necessarily too bad either, although recently the ServletOutputStream interface has been more involved to intercept properly.
I found doing this via PrettyPrinter problematic on earlier Jackson versions (such as your 2.4.4), in part because of the need to go through an ObjectWriter to configure it properly: only fixed in Jackson 2.6. For completeness, this is a working 2.5 solution:
#Test
public void append_newline_after_end_of_json() throws Exception {
// Jackson 2.6:
// ObjectMapper mapper = new ObjectMapper()
// .setDefaultPrettyPrinter(new NewlineAddingPrettyPrinter())
// .enable(SerializationFeature.INDENT_OUTPUT);
// ObjectWriter writer = mapper.writer();
ObjectMapper mapper = new ObjectMapper();
ObjectWriter writer = mapper.writer().with(new NewlineAddingPrettyPrinter());
assertThat(writer.writeValueAsString(ImmutableMap.of()), equalTo("{}\n"));
assertThat(writer.writeValueAsString(ImmutableMap.of("foo", "bar")),
equalTo("{\"foo\":\"bar\"}\n"));
}
public static final class NewlineAddingPrettyPrinter
extends MinimalPrettyPrinter
implements Instantiatable<PrettyPrinter> {
private int depth = 0;
#Override
public void writeStartObject(JsonGenerator jg) throws IOException, JsonGenerationException {
super.writeStartObject(jg);
++depth;
}
#Override
public void writeEndObject(JsonGenerator jg, int nrOfEntries) throws IOException, JsonGenerationException {
super.writeEndObject(jg, nrOfEntries);
if (--depth == 0) {
jg.writeRaw('\n');
}
}
#Override
public PrettyPrinter createInstance() {
return new NewlineAddingPrettyPrinter();
}
}
Not yet tested but the following should work:
public class MyObjectMapper extends ObjectMapper {
_defaultPrettyPrinter = com.fasterxml.jackson.core.util.MinimalPrettyPrinter("\n");
// AND/OR
#Override
protected PrettyPrinter _defaultPrettyPrinter() {
return new com.fasterxml.jackson.core.util.MinimalPrettyPrinter("\n");
}
}
public class JerseyConfiguration extends ResourceConfig {
...
MyObjectMapper mapper = new MyObjectMapper();
mapper.enable(SerializationFeature.INDENT_OUTPUT); //enables pretty printing
// create JsonProvider to provide custom ObjectMapper
JacksonJaxbJsonProvider provider = new JacksonJaxbJsonProvider();
provider.setMapper(mapper);
register(provider); //register so that jersey use it
}
Do not know if this is the "cleanest" solution but it feels less hacky than the others.
Should produce something like
{\n "root" : "1"\n}\n{\n "root2" : "2"\n}
But it seems that does not work if there is only one root element.
Idea is from https://gist.github.com/deverton/7743979
I created a hadoop multinode cluster and my target is to know processor information of all nodes in the cluster by running jar file in master.I have the code to know processor info in a node.How can i run this code in all nodes and retrieve the processor information using java.
thanks in advance for help..
Driver code:
Here i have used only mapper class
public class ProcInfoCluster extends Configured implements Tool {
#Override
public int run(String[] arg0) throws Exception {
// TODO Auto-generated method stub
if(arg0.length<2)
{
System.out.println("please provide the output file paths");
return -1;
}
JobConf conf=new JobConf(ProcInfoCluster.class);
conf.setJobName("ProcesorInfo");
FileInputFormat.setInputPaths(conf, new Path(arg0[0]));
FileOutputFormat.setOutputPath(conf,new Path(arg0[1]));
conf.setMapperClass(ProcInfoMapper.class);
//conf.setReducerClass(WordReducer.class);
conf.setMapOutputKeyClass(Text.class);
conf.setMapOutputValueClass(Text.class);
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(Text.class);
JobClient.runJob(conf);
return 0;
}
public static void main(String args[])
throws Exception
{
int exitCode=ToolRunner.run(new ProcInfoCluster(), args);
System.exit(exitCode);
}
}
Mapper code:
Here in this mapper code i have written code to know processor information and tried to send it as key value pairs
public class ProcInfoMapper extends MapReduceBase implements Mapper<LongWritable, Text, Text,Text>
{
#Override
//map function to retrieve processor information of all nodes
public void map(LongWritable key, Text value,
OutputCollector output, Reporter reporter)
throws IOException {
// TODO Auto-generated method stub
String no=System.getenv("PROCESSOR_ARCHITEW6432");
//to know processor info
System.out.println(no);
String s=new String("ProcInfo");
output.collect(new Text(s), new Text(no));
}
}
and i had a doubt that whether we can write a map reduce application only with mapper class without the reducer class