Verbose

I do most of my development in python and scala these days. I totally forgot how verbose java is until I saw some Apache Spark coding examples.

Code in python

file = spark.textFile("hdfs://...")
errors = file.filter(lambda line: "ERROR" in line)
# Count all the errors
errors.count()
# Count errors mentioning MySQL
errors.filter(lambda line: "MySQL" in line).count()
# Fetch the MySQL errors as an array of strings
errors.filter(lambda line: "MySQL" in line).collect()

The same code in Scala:

val file = spark.textFile("hdfs://...")
val errors = file.filter(line => line.contains("ERROR"))
// Count all the errors
errors.count()
// Count errors mentioning MySQL
errors.filter(line => line.contains("MySQL")).count()
// Fetch the MySQL errors as an array of strings
errors.filter(line => line.contains("MySQL")).collect()

And finally in Java:

JavaRDD<String> file = spark.textFile("hdfs://...");
JavaRDD<String> errors = file.filter(new Function<String, Boolean>() {
  public Boolean call(String s) { return s.contains("ERROR"); }
});
// Count all the errors
errors.count();
// Count errors mentioning MySQL
errors.filter(new Function<String, Boolean>() {
  public Boolean call(String s) { return s.contains("MySQL"); }
}).count();
// Fetch the MySQL errors as an array of strings
errors.filter(new Function<String, Boolean>() {
  public Boolean call(String s) { return s.contains("MySQL"); }
}).collect();

Twice as many lines of code.

Advertisements

Installing java and browser plugin

I’ve spend some time today trying to figure out how to enable the chromium browser to run the latest version of sun..erh, oracle java. Apparently chromium reads the .mozilla/plugins folder… This i how I got things to work

Installing java

Download a java 7 without an installer from java.com. Untar this in /usr/local

# cd /usr/local
# sudo tar xzf ~/Downloads/jre-7u13-linux-x64.tar.gz

And update alternatives:

# sudo update-alternatives --install /usr/bin/java "java" /usr/local/jre1.7.0_13/bin/java 1062
# update-alternatives --config java

Install the Firefox Plugin

# mkdir -vp ~/.mozilla/plugins

32-Bit version:

# ln -s /usr/local/jre1.7.0_13/lib/i386/libnpjp2.so ~/.mozilla/plugins/

64-Bit version:

# n -s /usr/local/jre1.7.0_13/lib/amd64/libnpjp2.so ~/.mozilla/plugins/