A sneak peek into Apache James

I’ve been playing around with Apache James, a complete email server written in Java, in the last few months and I’d like to share some of my thoughts about it. (I’ll focus on version 2.3.2 as the 3.0 version is still in beta.)

In short
Apache James is a flexible, highly configurable and extendable email server (SMTP, POP3, IMAP4 [since James 3.0]). It can store emails and users either in the file system or in a database (e.g. MySQL, PostgreSQL, and others) and handles emails for multiple domains via mappings.

Some insights
There is a comprehensive documentation at the James website including a wiki, so I will not go too much into detail here. Some of the server’s main concepts are storage, processor, matcher and mailet.

Storage
Both emails and users (and other configuration data) is persisted in the storage. James comes with a filesystem-based and a JDBC-based storage implementation, but other types of storage are possible too (e.g. LDAP, JPA and others, some of them will be provided with version 3.0).

Processor
When an incoming email is processed by the server it is injected into a so-called processor, which is a kind of queue consisting of actions to be performed on the email. Those actions are performed by the mailets.

Mailets
A mailet is similar to a servlet in a web container (like Tomcat), but it handles an incoming email instead of an HTTP request. James comes with many ready-to-go mailets, but it is very easy to implement own mailets to perform special tasks.

Matcher
A matcher decides if a a mailet will be executed for an incoming email. It checks if a condition is met and tells the email engine to execute the associated mailet.

Making it work all together
Sounds all a little bit confusing? Actually it isn’t. This is how everything works together:

  • The email engine receives an incoming email, wrapping it internally with an container (which provides some helpful functions)
  • The email engine forwards the the wrapped email to the processor named “root”
  • The “root” processor consists of a sequence of mailets, each having a matcher associated.
  • The mailet’s matcher is executed, returning whether or not the mailet should be executed.
  • The matcher is then executed (or maybe not). It will “do something” with the email, e.g. a virus check, some validation, changing data, forwarding or rejecting or discarding the email, putting it into another processor, or doing some other action.
  • If the email was not discarded, then the next mailet is processed, then the next, and so on.

Actually, there is a lot more going on internally, and in many points I oversimplified things here, but I just to give you a first impression about how James works.

Why is this so great (or useful)?
For me the most interesting point is that you can completely configure and extend the way how an incoming  email is processed. You can directly interact with an incoming email: you can convert its content or forward it to a custom script or handle it any other way that you can imagine.

One example? Sure!
The following mailet adds support for an address mapping to receive emails with multiple address suffixes into one account: an email address like jcoder+projectone@example.com
will become jcoder@example.com (and will be delivered to the account “jcoder” then.)

package me.jcoder.blog.examples.james.mailet;

import java.util.Collection;
import java.util.Iterator;
import java.util.LinkedList;
import javax.mail.MessagingException;
import org.apache.mailet.GenericMailet;
import org.apache.mailet.Mail;
import org.apache.mailet.MailAddress;

/**
 * Mailet to support the "plus sign alias" syntax in addresses, so that
 * name+something@domain becomes name@domain
 * @author jCoder
 */
public class PlusAliasSupport extends GenericMailet {
    private static final String MAILET_NAME = "PlusAliasSupport";

    @Override
    public void service(Mail mail) throws MessagingException {
        if (mail == null) {
            return;
        }
        try {
            Collection recipients = mail.getRecipients();
            Collection recipientsToRemove = new LinkedList();
            Collection recipientsToAdd = new LinkedList();
            for (Iterator itr = recipients.iterator(); itr.hasNext(); ) {
                MailAddress address = (MailAddress)itr.next();
                if (address == null) {
                    continue;
                }
                String localPartStr = address.getUser();
                int plusIndex = -1;
                if ((localPartStr != null) && ((plusIndex = localPartStr.indexOf('+')) > -1)) {
                    String newLocalPart = localPartStr.substring(0, plusIndex);
                    MailAddress replaceAddress = new MailAddress(newLocalPart, address.getHost());
                    recipientsToRemove.add(address);
                    recipientsToAdd.add(replaceAddress);
                }
            }
            recipients.removeAll(recipientsToRemove);
            recipients.addAll(recipientsToAdd);
        } catch (Exception ex) {
        }
    }

    @Override
    public String getMailetName() {
        return MAILET_NAME;
    }
}

It’s pretty helpful to get the source code of James to take a lokk at how the provided mailets and matchers are implemented.

Some notes and tipps
When writing matchers and mailets or running James as your (test) email server, there might be some pitfalls, so I’ll share some of the points that I came across.

  • When writing matchers and mailets you might want to download the Mailet SDK from the James website. This is generally a good idea, but make sure that you get the right version! James 2.3.2 normally uses the Mailet 2.3 API and the website provides the 2.4 API, so things will not work. If in doubt, add the mailet API-related JARs from a James installation.
  • When placing the JAR into the ${JAMES_DIR}/apps/james/SAR_INF/lib folder, don’t forget to add your mailet/matcher package name to the mailetpackagesmatcherpackages section of the config.xml file so it can be found by the server.
  • If you are going to use the JDBC-based storage, you need to put the JDBC driver for your database in a directory where the server can find it. It might not be best-practise, but I suggest to put the JDBC driver JAR into ${JAMES_DIR}/lib where it is accessable by the main server.
  • The same goes for using TLS/SSL. If you want to provide secure POP3 access, then you not only need a certificate (which is a whole other story to maybe being covered in an other posting), but you might run into some SSL-socket-related errors. One solution is to copy the sunjce_provider.jar from your Java runtime into ${JAMES_DIR}/lib as well.
I hope that this posting gives you a first impression about the flexibility of the email server James. If you have any questions or ideas, feel free to leave a comment.

Networking with the FEZ Panda II

First of all: using the ethernet connection with the FEZ Panda II works well if you follow the steps from GHIElectronic’s eBook “FEZ Internet of Things”. You just set up MAC, IP, netmask, gateway and DNS server settings and you are ready to go.

But, there is actually a drawback: although the FEZ Panda II uses the .NET Microframework 4.x (NETMF), it does not fully support the System.Net namespace due to resource limitations of the device!

This means that all code using System.Net from the NETMF will not work, giving you a System.NotSupportedException. One disadvantage of this is, that most 3rd party NETMF projects using networking will not work as well.

The workaround to this problem is to use the network classes provided by GHIElectronics. In detail, it helps to add the following references to your project:

  • GHIElectronics.NETMF.Net
  • GHIElectronics.NETMF.W5100
  • GHIElectronics.NETMF.W5100.Http

Those references provide all the classes used to successfully establish network connections via TCP or UDP. As said above, the examples in the eBook are very useful, so I’ll not go into detail here ;)

Now that I basically got the network connection working I’m going to think about some example usages and over the course of the next weeks I’ll share my observations (and maybe some code too) here.

Choosing a private MAC address

When doing some research for connection my Netduino to the local network (using the Connect Shield) I stumbled upon the fact that I not only need to set the IP adress, network mask and gateway, but also the device’s MAC address – something that you normally don’t need to do because the vendor of the network device burned a globally unique MAC address into the device.

While it should not be a serious problem to randomly generate a MAC address (which might interfer with a real device’s MAC address somewhere on the planet) I wanted to do it “the right way”. There are many hints on the internet how to choose a private MAC address, mainly for use in virtual computers and many information about the pattern how MAC addresses for vendors are built.

So, to cut a long story short, I’ve come to the conclusion that it can be considered safe to use a MAC address starting with 01 followed by a random combination. I’ll give it a try and if it doesn’t work, then I’ll do some more in-depth research on the topic ;)

Next: external power supply and SD card

After taking a short break with Netduino hacking I’ll start over on the next days to test an external power supply that I’ve bought some days ago. It has an adjustable volatge range from 3 to 12 V and comes with a variaty of connectors. This external power supply is intended to support projects to be run independently of an USB power source of a computer.

I also got two micro SD cards which hopefully will work with the Netduino and which will allow to store a decent amount of runtime data.

Over the course of the next days I’ll try to get the two things mentioned above up and running and as always I’ll share the results of my tests with you.

Update: the power supply works pretty well.

New piece of hardware: an LCD

Guess what I found at my desk when I came to the office today?

Saying thank you to my coworker Oliver for long-time renting me this 2×24 LCD with a HD44780 controller. I’m afraid that it’ll take some weeks to get it up and running, because currently I don’t have any breadboards, cables or things like that.

Netduino Switching Lights (with interaction)

After my first and very simple Netduino project, the traffic lights, I wanted to get a little step further. So I’ve put together some lines of code to make the LEDs show the sequence red, yellow, green, off with a fixed interval in between. And I want this interval to be controllable with the Variable Resistor (POT).

The main goal was actually not to build some great light animation, but to learn more about the possibilities behind the scenes. You’ll find find the complete project in the attached ZIP file (see end of this posting), so I’ll focus on two simple, but helpful details in this posting.

The first one is that I encapsulated the access to the LEDs using a class which holds a reference to the output pin, the LED’s state (on/off) and provides the three methods On(), Off() and Toggle() to make it easier to interact with the LED in code. I’m pretty sure that I’ll use (and extend) this LED class in upcoming projects.

public class LED : IDisposable
{
    private OutputPort port;
    private bool state;

    public LED(FEZ_Pin.Digital pin)
        : this(pin, false)
    {
    }

    public LED(FEZ_Pin.Digital pin, bool initialState)
    {
        this.port = new OutputPort((Cpu.Pin)pin, initialState);
        this.state = initialState;
    }

    public void On()
    {
        this.port.Write(true);
        this.state = true;
    }

    public void Off()
    {
        this.port.Write(false);
        this.state = false;
    }

    public void Toggle()
    {
        if (this.state == true)
        {
            this.Off();
        }
        else
        {
            this.On();
        }
    }

    public bool State
    {
        get
        {
            return this.state;
        }
    }

    public void Dispose()
    {
        this.port.Dispose();
    }

}

The second class simplifies the Analog Input handling. When attaching the POT you normally create a static variable to hold the reference to the analog input and read its value when you think it might have changed and you then need to react to this change.

This might be a practical approach in many cases but sometimes it would be nice to be more event-based. So that’s where the class ActiveAnalog kicks in. It encapsules an analog input pin offers the event ValueChanged which provides the current value and the offset change from the previous value. Internally a thread is used to read the analog pin’s value every 20 milliseconds, compare the value with the previous value and fire the event if the value has changed. Nothing spectacular, but it makes it easier for some projects to react on analog pin value changes.

public class ActiveAnalog : IDisposable
{
    private const int READ_INTERVAL = 20;
    private AnalogIn analogPin;
    private Thread readThread;
    private bool active = true;
    private int previousValue;
    private int currentValue;

    public delegate void AnalogValueChange(int currentValue, int offset);

    public event AnalogValueChange ValueChanged;

    public ActiveAnalog(FEZ_Pin.AnalogIn pin)
        : this(pin, 100)
    {
    }

    public ActiveAnalog(FEZ_Pin.AnalogIn pin, int maxScale)
    {
        if (maxScale < 1)         {             maxScale = 1;         } else if (maxScale > 3300)
        {
            maxScale = 3300;
        }
        this.analogPin = new AnalogIn((AnalogIn.Pin)pin);
        this.analogPin.SetLinearScale(0, maxScale);
        this.readThread = new Thread(this.ReadValue);
        this.readThread.Start();
    }

    private void ReadValue()
    {
        this.currentValue = this.analogPin.Read();
        while (this.active)
        {
            this.previousValue = this.currentValue;
            this.currentValue = this.analogPin.Read();
            if (this.previousValue != this.currentValue)
            {
                if (this.ValueChanged != null)
                {
                    this.ValueChanged(this.currentValue, this.currentValue - this.previousValue);
                }
            }
            try
            {
                Thread.Sleep(READ_INTERVAL);
            }
            catch (Exception)
            {
            }
        }
    }

    public int CurrentValue
    {
        get
        {
            return this.currentValue;
        }
    }

    public int PreviousValue
    {
        get
        {
            return this.previousValue;
        }
    }

    public void Dispose()
    {
        this.active = false;
        this.readThread.Abort();
        this.readThread = null;
    }
}

The complete project is available for download at http://blog.jcoder.me/files/Netduino/NetduinoSwitchingLights.zip

Continuous project: C# Extension Methods

There are a lot of cool features in the current C# version (and I’m sure that I will cover most of them in future postings), but one of the features I like most are Extension Methods.

If you’ve never heard about Extension Methods, let me cite the MSDN documentation on Extension Methods:

Extension methods enable you to “add” methods to existing types without creating a new derived type, recompiling, or otherwise modifying the original type.

Extension Methods are a very powerful feature when it comes to writing easy-to-understand code. So, over the course of the weeks and months I’ll post code fragments with (mostly) helpful Extension Methods, at least if they’re helpful for me. I will also upload the latest source code files to my blog.

Unless otherwise stated the code is provided as “Public Domain” which means that you can modify and use it in all your projects.

So, let’s get started with the first methods.

/// <summary>
/// Performs an action with each of the elements in this IEnumerable
/// </summary>
/// <param name="enumerable">this IEnumerable</param>
/// <param name="action">action to be performed on each element</param>
/// <returns>IEnumerable</returns>
public static IEnumerable<T> WithEach<T>(this IEnumerable<T> enumerable, Action<T> action) {
	if (action == null) {
		action = (obj) => {};
	}
	foreach (T oneObject in enumerable) {
		action(oneObject);
		yield return oneObject;
	}
}

/// <summary>
/// Concatinates all elements of this IEnumerable to a flat string, optionally quoting them
/// </summary>
/// <param name="enumerable">this IEnumerable</param>
/// <param name="glue">glue between the elements</param>
/// <param name="quoteStart">quote at start of each element</param>
/// <param name="quoteEnd">quote at end of each element</param>
/// <returns>flat string</returns>
public static string ToFlatString<T>(this IEnumerable<T> enumerable, string glue, string quoteStart, string quoteEnd) {
	if (glue == null) {
		glue = ",";
	}
	if (quoteStart == null) {
		quoteStart = "";
	}
	if (quoteEnd == null) {
		quoteEnd = "";
	}
	StringBuilder strBuilder = new StringBuilder();
	bool useGlue = false;
	foreach (T oneObject in enumerable) {
		if (useGlue == true) {
			strBuilder.Append(glue);
		} else {
			useGlue = true;
		}
		strBuilder.Append(quoteStart).Append(oneObject).Append(quoteEnd);
	}
	return strBuilder.ToString();
}

But wait, there’s more ;)

public static string ToFlatString<T>(this IEnumerable<T> enumerable, string glue)
public static string ToFlatString<T>(this IEnumerable<T> enumerable, string glue, Func<T, string> toStringFnc)

As don’t want this posting to be just code, I’ve only outlined the additional method signatures here, so feel free to download the complete file from http://blog.jcoder.me/files/csharp/IEnumerableExtensions.cs

If you have any questions or ideas for useful Extension Methods, please leave a comment below.