The Intellog Blog

The previous post described the proposed Display class, which is intended to wrap a stable layer of code around small form factor displays which will be used in Arago and similar products.  It only makes sense to have this class easily extendable, in some way, for new displays incorporated into future designs.  This will be accomplished through the use of Java .properties files, wherein information specific to each display will be stored in its own file — shades of termcap!  By convention, the unique string of characters associated with the display will form the root of the name and will be suffixed by .properties of course.  For example, the GLK24064-16-1U currently being use would have a properties file called GLK24064-16-1U.properties.  These files will all be collected up in one directory, which for the time being is located within the Intellog project, in the directory src/intellog/display.

An article in JavaWorld by Vladimir Roubtsov was the basis of the implementation strategy, wherein Java’s ClassLoader is used to find the resource — that is, the .properties file.  It subsequently uses InputStream to populate an instance of the Properties class.  As Vladimir says in his article, this is the preferred approach because "classloaders essentially act as a layer of abstraction between a resource name and its actual location on disk".  In other words, it will make for reduced cost-of-ownership down the road — easier packaging and deployment, for instance.  In order to distill it to its essence, five lines were extracted from the code in Vladimir’s article, and refactored into a Util.loadProperties(string) method.  This takes the name of the resource (eg. GLK24064-16-1U.properties) as input, and returns an instance of Properties populated with all of the properties found in the file

It then became a matter of establishing some conventions for the content of each .properties file, which didn’t have to be anything fancy.  The properties are generally named to reflect the methods to which they are related.   For instance, the line GLK24064-16-1U.properties relating to the display’s brightness, not surprisingly, uses a property named brightness, which reflects the setBrightness(int) method.  Currently, properties are lumped into two categories; specifications, which identify the characteristics of the display, and control sequences, which contain the strings of control characters which produce the desired effect.  With the latter, it’s tempting to try and support every conceivable format for the control information, but the decision was made to use just one — the 0x00 format, separated by spaces.

Having cracked the problem of how to store the display-specific information in a .properties file, it was an obvious choice to use the same approach to store the Arago-specific properties as well.  These types of properties are going to be stored in arago.properties, and its content is retrieved using the same Until.loadProperties() method described above.  Also, one final note — to drastically streamline the code, it’s assumed all of these properties files will be read-only.  If they need to be maintained, that will be done so by hand, using any handy text editor.

Code Shavings  Given this is the first real Java code to get cut in a while, it’s still uncertain exactly what part of it is general in nature, and what part is specific to the Arago.  To reflect this, two Java projects were created in Eclipse; Intellog, for code with general application, and Arago for code specific to, you guessed it, the Arago.   ♦   The Java Tutorials continued to be a very useful resource, and the unit entitled Properties was particularly useful in working through the decision making process described above.  ♦   Instinct (and little else) dictated XML would be the correct format in which to store the display-specific information.  However, Bart Busschots makes a pretty good case for going with the Java .properties flow and using the very simple format it specifies.  And truth be told, there is a very easy way to convert them to XML in the future, if  necessary.

With a specific display chosen, it was time to undertake the development of an interface between the physical hardware and the application code, which will be pure Java.  The approach is to use an abstraction layer to enable the change out of display hardware with minimal impact on the rest of the application code. 

To avoid repeating the mistake of inventing something already invented, a quick survey of the landscape resulted in at least one class somewhat along the same lines; Karl-Petter Åkesson’s TiniLCD.  It has reasonably good functionality, but has been more-or-less dormant since 2000 and is quite specific to the TINI Network Microcontroller.  Two related threads (this and this one) were found on the Matrix Orbital (MO) forums, as well as some sample code, but they did not provide a complete, coded solution either.  Finally, a post was made to the MO forums to solicit others ideas, and relevant responses will be posted as comments below.

Reasonably satisfied it was untilled soil, work began on a proposed class tentatively named Display.  Also proposed are the methods required for the minimum functionality for the first iteration of Arago development.  All member names are tentative as well;

Constructor

• Display(String port, String type)  For now, it’s assumed the display will be accessed through a serial port, and hence the need to supply a communications port identifier (eg. COM3, COM4 etc.).  type identifies the specific kind of display.  Initially, this will be limited to the single display selected for this project, which is the GLK24064-16-1U, but others will be added, as needed (or as requested by third-party users of this class).

Methods (alphabetical order)

• clear()  Clears the display, and moves the cursor home, which is to say the top left hand corner.  This is deemed to be row zero, column zero.
• getAutomaticScroll()   The state of the automatic scrolling capability of the display, either on (true) or off (false).
• getBacklight()  The state of the backlight feature, either on (true), or off (false).  If backlight capability is not supported in hardware, returns false.
• getBrightness()  The current setting of brightness value, zero through 100.   See setBrightness() for a description of the brightness scale.
• getPort()  Read-only communications port identifier name (eg. COM4).  Can only be set with the constructor at instantiation.
• getType()   Read-only type name associated with the display (eg. GLK24064-16-1U).  Can only be set with the constructor, at instantiation.
• getColumn()  Read-only.  The number of columns available on the display.
• getRow()  Read-only.  The number of rows available on the display.
• println(String text)  Writes text to the display, starting at the current cursor position.  An emulation of the standard Java System.out.println method.
• println(String text, int column, int row)  Writes text at the specified column and row of the display.  Both column and row are zero-based. Throws an exception if either values are greater than the values returned by getColumn() or getRow() above.  
• setAutomaticScroll(bool state)  Sets the automatic scrolling feature of the display to either on (true) or off (false).  If the automatic scrolling feature is not implemented in hardware, then setting to true is ignored.
• setBacklight(bool state)  Sets the backlight feature of the display to either on (true) or off (false).   If the backlight feature is not implemented in hardware, then setting to true is ignored.
• setBrightness(int amt)  Sets the brightness of the display to a relative value in the range 0 through 100, or in other words, ‘percentage on’.  This doesn’t necessarily reflect the full resolution of values for a typical display, which typically run 0 through 255, but is deemed sufficient, and is less hardware-specific.

Of course, the temptation is to go on and on, but these members reflect the basic requirements are they are understood at this moment.  Of course, anyone who is interested may post comments or suggestions for the class, and they will be incorporated on a best efforts basis.  Also, if you have a specific display you would like to see supported, leave a comment with a link to some technical data.

In Choosing a Keypad and Interface, there was a "blinding flash of the obvious" wherein the four pins associated with each of the four keys on PLX series keypad could be used to drive four digital inputs on the 8/8/8 interface*.  By associating a software listener with each of these inputs, it should be possible for the running code to determine if and when a function key has been pressed and respond accordingly.

The next step in the pursuit of this objective was to validate the operation of the 8/8/8 and the keypad separately.   With the 8/8/8, it required the use of the Phidget Control Panel software and a short length of 24AWG wire.  One end of the lead was attached to the G (ie. ground) connector for the input block.  The other end of the lead was then touched to each of the other connectors on the input block, and as expected, each of the digital inputs was triggered.  Click the image above to see an example of digital input #3 when it is connected.

Similarly, the keypad was tested using a multimeter, testing for continuity between pins.  Facing the back of the keypad (with F1 up and F4 down) the right-most pin is common.  The pin immediately to its left maps to F4, then F3, and so on.  Press a key, and there’s continuity between ground and the pin associated with that particular key.  In other words, they’re four push-to-close switches — it’s just that simple.

So, pending actually hooking it up so the pins of the keypad are connected to the digital inputs of the 8/8/8, it can be assumed that the four function keys will present themselves in the software as one simple digital input per key.

*Or any interface supporting digital input, for that matter.

If at first you succeed, try and hide the surprise.  So it was when it came time to put the theory described in Powering Up the Display to a practical test.  The objectives were pretty simple.  Adapt a 12V, 2.0A wall wart (temporarily standing in for a lead-acid battery) to the 5V required by the GLK24064-16-1U LCD display.  Then, use a USB-to-DB9 adapter to connect a USB port on the development laptop to the DB-9 on the display to feed it text and other control information.  If all were to go as planned, it should have been possible to send text to the display.  As the photograph to the left illustrates (click for for larger image), mission accomplished.

The heart of the exercise was transforming the 12V input to 5V output.  This was the job of the PT78ST100-series integrated switching regulator (ISR).  It can be seen in the photograph above to the immediate left of the red lead going to the display.  Using a common, garden variety breadboard, it was wired up according the schematic to the right (click for larger image), which was found in the PT78ST105* data sheet.  Power from the 12V wall wart were run to the Vin and COM connections, and the output 5V from Vout and COM to the display via a Tyco MTA-100 connector (Digi-Key A31019-ND).  The leads were set in the connector without solder using the handy (and relatively cheap) T-handle tool (Digi-Key A9982-ND).  The circuit was powered up — no smoke was evident — and the display lit up and displayed the MO logo exactly as expected.  Also, the PT78ST105 should be able to handle all the power 5V power requirements of the completed device, not just the display, so at least a couple of birds have been killed with this one stone.

Making the USB-to-DB9 with the Future Technology Devices International (FTDI) US232R-10 turned out to be just slightly more difficult.  Not so much with the wiring up — it’s just a matter of plugging it in, after all — but rather with the installation of the drivers.  These were downloaded from the FTDI driver page, but any attempt to install them using the USB installation wizard failed with a missing file message of some sort.  However, by downloading the setup executable from the same page and running it, the USB-to-DB9 adapter configured itself correctly at COM4.  The only specific change of properties required was to set the baud rate to 19200, which is the default for the LCD panel.  First attempts to send text to the display using echo 'test' >"com4:" from the DOS command prompt did not budge the MO logo from the display, though.

The missing link turned out to be the download of MO’s MOGD# utility.  The first reading of the manual makes it sound like a utility for uploading fonts, bitmaps and other maintenance functions, but it’s actually a fairly comprehensive configuration and test utility.  By navigating to the test text section, it was possible to direct text to the display as shown in the photograph above.  The only other thing MOGD# was used for was to set the default baud rate for the display to 115,200, the fastest possible speed.  By configuring the serial port to match that, it was possible to communicate with the display at this faster speed, which makes an easily noticeable difference.

Code Shavings  Julian at MO informed me that the USB-to-DB9 adaptation really wasn’t necessary.  There is a new version of the GLK24064 display which has direct USB input, which is not yet on their website.  ♦  Turns out I ordered the wrong capacitor for C1 in the schematic above — instead of 1 µF capacitor called out, I somehow wound up with 0.1 µF.  The schematic noted C1 as optional, however, so it was simply omitted from the prototype for the time being.  There didn’t appear to be any negative consequences.  ♦  The price of ceramic capacitors is related to the voltage to which they are rated.  For example, 1.0 µF capacitors rated to 50V run $1.25, whereas the same capacitor rated to 25V is $0.32, about a quarter of the cost.  But since the PT78ST105 is capable of regulating 9V through 38V, it makes sense to have a capacitors in excess of the maximum voltage — right?

*The PT78ST105 is simply the 5V version of the PT78ST100 series.  Other voltages are supported, each with their own unique suffix character.

After some reasonable consideration, a four key keypad from Storm Interface was chosen as a good place to start for operator input.   Specifically, the four key PLX series unit (Digi-Key MGR1541-ND).  It’s not indestructible, but very rugged, and certainly fine for the time being.  The general objective is to implement soft key functionality; function keys which produce different actions depending on where the user is located in the application navigation scheme.  More than likely, the current definition of the keys will be displayed on the LCD panel at all times, for easy reference by the user.  It was certainly one of the reasons for choosing a fairly large panel with its attendant cost.

Trickier was the problem of how to interface the keypad to the CPU.  Inspection of the back of the keypad reveals five pins.  It’s easiest to think of the keypad as a series of four switches which map onto these pins.  By sensing whether a connection is being made on a given pin, it’s possible to determine which key has been pressed.  The first inclination was to try and find some sort of adapter where one end would plug into the back of the keypad, and the other into a serial port on the host computer.  Thence, said adapter would transmogrify the pressing of keys into a stream of ASCII characters.   It would then simply be a matter of monitoring the serial port with some sort of software listener and trigger an appropriate application response depending on which key had been pressed.

Such adapters — or something pretty close to it — exist, all right, not the least of which from Storm themselves.  They are also available from others, as well, such as NCD and Solutions Cubed.  There is even a chip dedicated to this task — the Fairchild MC74C922.  But for reasons which were initially obscure, they are for 12 or 16 key units exclusively.   In a blinding flash of the obvious, it was suddenly apparent decoding four keys just doesn’t make sense.  Tie each of the four keys to their own digital input, put a software listener on each one, and it’s job done.  At least in theory it should be cheap (both in terms of money and power), if not particularly elegant.  There are digital inputs a plenty on the 8/8/8 sensing and control interface already specified in the design.

Code Shavings  Thanks to Ryan Coggins at Storm Interface for his patience in answering all of my rather mundane questions.  I owe him a coffee the next time I’m in his neck of the woods.

A while back a decision was made regarding the integrated graphics display, the winner being Matrix Orbital GLK24064-16-1U.  It was time to fire it up and begin to explore its capabilities both to display text messages as well as simple graphic elements such as line or bar charts.  Initially, there were two connectors which were of concern; a DB-9 serial cable connection (#4 in the photograph at the top left) which is used for communicating with the display, and the MTA-100 (#1) which is used to supply power.  The seemingly simple task of making these two connections required some research, on both counts.

In the case of a DB-9, it was relatively simple; adapt one of the USB ports on the development mule laptop to the DB-9 configuration.  There are a lot of different products to do this, but the one chosen was the Future Technology Devices International (FTDI) US232R-10 (Digi-Key 768-1013-ND).  In conjunction with the driver software, the USB port into which it is plugged turns into a virtual communications port (VCP).  Mission accomplished, subject to actually plugging it in and confirming it works as advertised.

It was slightly more problematic in the case of the MTA-100 power connector, which required quite a bit more  detective work.  This started with a thread on the Matrix Orbital support forum, where the part number (Digi-Key A1922-ND) for the board-side connector was offered up.  Digi-Key does a great job in providing mating connector information, but unfortunately, there were no less than 89 different connectors which could fit!  So the hunt for the right connector wasn’t done yet. 

Narrowing down the choices to just one was first tackled by determining (then eliminating) any connector with either lead (non-RoHS compliant), or gold, which was deemed superfluous to requirements at this early stage.  This trimmed down the list of possibilities quite considerably.  It was trimmed still further by selecting the specific wire gauge which would be used; taking the published power consumption for the display; 45 mA base, plus 75 mA for the backlight, plus 20 mA for the General Purpose Output (GPO) this resulted in a total draw of 140 mA, at 5V, or 0.7 watt*.  Assuming 1.5 watt (double the original power estimate), and a maximum six foot cable run with a 10% cable loss, the handy wire gauge calculator from Denning Electronics specified a minimum of 30 AWG.  Very tiny, indeed, as might be expected.

Then, another look at the Digi-Key mating connector info showed connector colours correlated to wire gauge, and the white connector — the ones which seem to be evidence most of the time — were 24AWG.  This locked it in — certainly more than required by the display, but obviously applicable to a wider variety of hookup situations which will undoubtedly be encountered in the near future.  At the very least, there was some factual basis which proved that 24AWG was not going to be under spec.

That still left a few choices; permutations and combinations of closed end versus feed through, and with or without polarizing keys.  Closed end was chosen more-or-less at random, and polarizing keys were chosen because of the dire warnings in the panel documentation about hooking up the power backwards.  This resulted in the final selection of A31019-ND, the closed end connector with polarizing tabs.  Subsequent test fitting indicated it was exactly the right connector for the job.  At the same time, the MTA-100 T-handle tool (A9982-ND) was obtained, for press fitting the wires into the connector without the aid of expensive, automated equipment.

*Calculating wattage is pretty simple — voltage multiplied by current, but for those who prefer the ‘certainty’ of a calculator, check out Service Matters offering in this regard.

In the previous post, the view PairRange was introduced as the mechanism to transform the Pair table — which consists of related pairs of values xAmt and yAmt — into lowerXAmt, upperXAmt, lowerYAmt and upperYAmt respectively.   A sensor value, found in Poll.amt for example, can then be compared to lowerXAmt and upperXAmt to determine into which ‘bucket’ it falls.  The relative position of Poll.amt between these two boundaries can then be used to do linear interpolation between the lowerYAmt and upperYAmt values.  However, the previous post also pointed out some "odd behaviour" of this view, wherein it only seemed to work if id_Correlation value was specified literally.

After a half-a-day or so of trying, it was finally determined the stated "odd behaviour" simply could not be overcome without recoding the PairRange view somewhat.  The main change was to ensure the omission of the WHERE clause resulted in all PairRange instances being displayed, as opposed to none, as was previously the case*.  This partially addressed the problem, but then even more weird behaviour was in evidence when specifying values for id_Correlation to look at subsets of PairRange.  For example, the syntax SELECT * FROM PairRange WHERE id_Correlation = 2 yielded no rows, whereas SELECT * FROM PairRange WHERE id_Correlation IN (SELECT 2) would return rows where id_Correlation = 2 as expected.  These are functionally identical statements, which should return exactly the same results — and yet they don’t.  This was beginning to look like a really peculiar flaw in the PairRange logic, or maybe even a subtle bug in H2.

Not wanting to get bogged down, the decision was made to use the PairRange view to create a temporary table _PairRange**.  H2 makes this particularly easy by permitting the syntax  CREATE TABLE _PairRange AS SELECT * FROM PairRange.  With the transformation of PairRange from view to table, all the weird behaviour described in this post and the previous one all went away.  It was then possible to write a single SQL statement which transformed all the related feeds identified in FeedDetail with their respective Correlation-based lookup and then aggregate their values.    This logic was captured in the text file refreshAggregateFeeds.sql.

*This was accomplished through the use of the CAST(SubQuery1.id_Correlation <> SubQuery0.id_Correlation AS BOOLEAN) AS lastFlg syntax, which eliminated the ‘boundary’ rows where id_Correlation changed from one row to the next.

**By convention, the leading underscore character is used to visually identify data which is denormalized in some way.

Most instances of Feed refer to a hardware sensor, and stores the periodically collected sensor values in related instances of Poll.  Some instances of Feed, however, are used to store information resulting from a calculation of some sort.  Initially, this is limited to the aggregation of values from several instances of Feed once the sensor values have been transformed to actual values with a Correlation-based lookup.  The relationship of which instance of Feed is the aggregator, and which instances of Feed are being aggregated, is maintained through the use of the newly-created FeedDetail table. 

In a given instance of FeedDetail, the id_Feed attribute identifies the aggregator Feed, whereas the id_Feed_related attribute identifies the feed being aggregated.  id_Feed must be populated, because all FeedDetail instances must be related to a Feed.  On the other hand, id_Feed_related is only populated when one Feed has a relationship to another.  In the future, other types of FeedDetail instances will be introduced which won’t have related instances of Feed, but rather, will have some other sort of calculation logic that needs to be applied.

Speaking of Correlation-based lookup, the first step for the logic was to create the PairRange view.  This takes subsequent instances of Pair, and creates lower and upper boundaries with them.  For instance, if xAmt and yAmt for two successive instances of Pair were populated with 0, 0 and 10, 15 respectively, then PairRange would have a single row with lowerXAmt, upperXAmt set to 0 & 10,  and lowerYAmt and upperYAmt set to 0 & 15 respectively.  This enables the easy comparison of values with Poll.amt to see where a particular value falls. The instance of PairRange also contains enough information to enable straight line interpolation between the lowerYAmt and upperYAmt.  For example, if Poll.amt is 4, the yAmt can be interpolated by taking 4 / (upperXAmt - lowerXAmt) * (upperYAmt - lowerYAmt) + lowerYAmt.  To complete the example, 4 / (10 - 0) * (15 - 0) + 0 => 0.4 * 15 + 0 which results in 6 for yAmt.

PairRange basically uses H2’s ROWNUM function to assign row numbers to two subqueries of Pair, and then joins them based on SubQuery1.seq - 1 = SubQuery0.seq AND SubQuery1.id_Correlation = SubQuery0.id_Correlation.  It works perfectly well, but does result in odd behaviour though; unless you query this view with a WHERE clause which specifies id_Correlation, it will only ever return rows for the first Correlation.  When H2 processes the view, the first time id_Correlation changes, the clause which is the basis for the join is no longer true, and because it’s not true, it’s also not true for any subsequent row of the view.  In other words, the breakage cascades its way though the balance of the view.  It’s just a little jarring to have things show up when you consciously specify something with a WHERE clause, and for things to disappear when you don’t, which is the reverse of what’s usually the case.

Code Shavings  Many thanks to Sergi Vladykin for his reply to my post on the H2 forum.  For a while there, I figured I was going to have to write some Java code to get around a limitation of H2’s ROWNUM function, but his suggestion worked like a charm and prevented all that.  ♦  One thing which got omitted from the previous post was the necessity for having the H2 Console open in order to have the JDBC code work.  This is because H2 Console starts the instance of H2 to which the JDBC code then connects.  The JDBC code, on its own, is not smart enough to start an instance of H2.

The time had come to tie together the ability to capture data from a sensor using Java, with the database structure described in recent posts.  Specifically, the objective was to poll the temperature sensor at specified intervals, and insert the captured value into Poll. A Sun-provided tutorial provided a great refresher on JDBC.  The only hitch in getting a simple JDBC-based application up and running was forgetting to add the H2 jar file into the CLASSPATH environment variable.

Then, it was necessary to modify OpenIFKitExample.java so it would take a value from the temperature sensor and insert the resulting value into Poll.  Once this was done, it was permitted to run on-and-off for an entire afternoon, which resulted in 8000+ instances of Poll being created.   Keep in mind, though, the value being polled from the sensor required conversion from a sensor value to an actual value.  This was accomplished through a JOIN with Correlation tentatively called writeTest.sql.  To completely close the loop, H2’s CSVWRITE function was used to export the results of writeTest.sql to a CSV file.  The latter was then imported into Excel, which was then used to produce the chart above (click for larger image).

This represents another fairly significant milestone, as the capture and storage of sensor information is fundamental to the overall design.  There are still many other aspects of this capability which need to be fleshed out, but until then, it’s good to know there are no major obstacles to implementing the necessary functionality.

Code Shavings  Some initial investigation of memory utilization was also undertaken.  As best could be determined — using Windows Task Manager, for lack of a better tool — initiating the process took about 15 MB of memory.  One it had been running for a while, the amount of memory crept down until it eventually stabilized around 10 MB while polling the temperature sensor once per second.

The previous post described the implementation of tables to support event scheduling.  The next objective was to establish logic to populate Journal based on descriptive information found in Event.  This resulted in the creation of the following objects;

• EventVerbose  A new database view, initially with two attributes; scheduleUdt and id_Event.  scheduleUdt contains the timestamp (ie. date/time) for the next occurrence of the event identified with EventVerbose.id_Event.  This timestamp is based on the timestamp of the previous occurrence of the event* plus the time period specified by the combination of Event.periodicAmt and periodicTlbl attributes.  For instance, if the previous timestamp was 2009-07-29 12:00:00, and periodicAmt and periodicTlbl were populated with 5 and mi respectively, EventVerbose.scheduledUdt would contain 2009-07-29 12:05:00.  In the case of the very first occurrence of a given event, then the current system time is used instead of the timestamp of the previous occurrence of the event.  (ie. ‘now’ plus five minutes.)
• insertEventSchedule.sql  This is a SQL statement (stored in a text file for the time being) which utilizes EventVerbose to determine when the next occurrence of an event is going to be, and inserts that information into Journal.  But here’s the trick; it only does this if this if the next occurrence of the event does not exist in Journal already.  If the next occurrence of the event does exist, insertEventSchedule.sql knows it should not insert the next occurrence into Journal a second time.  It does this primarily by looking for an instance of Journal which has scheduleUdt populated, but initiateUdt still empty.  In other words, an event has been scheduled to run, but has not yet started to run.
• updateEventSchedule.sql  This SQL statement is the counterpoint to insertEventSchedule.sql.  It finds instances of Journal where scheduledUdt is populated and initiateUdt is empty, and populates initiateUdt with the current system time.  Now, when insertEventSchedule.sql runs subsequently, it can no longer find the criteria for a scheduled event, and therefore inserts the next scheduled event into Journal.

*One would think the ‘timestamp of the previous occurrence of the event’ would be found somewhere in Event.  But one would be wrong.  It’s determined by taking the maximum (latest) scheduleUdt from the Journal table for a given Journal.id_Event.