Analog Presets v2.0 [ Jul 23 2019, 119.85 KB, MD5: ]
- Added the ability to control multiple module types in one level
Available in JANOS 1.8.
| Name | Version | Release Date | Size | MD5 |
|---|---|---|---|---|
| JMP Protocol Manual | v2.0 | Mar 14 2024 | 230.0 KB | 905f124811e438ea93ccd542bdd66f41 |
The JMP protocol shares the messaging JSON structure that is used by the DCP (Dynamic Configuration Page) and the web-socket protocol but on port 9220, by default.
To make processing easier we wrapped the JSON message in a two element JSON Array.
[ number, object ]
Where number is the size in bytes/characters of the JSON object that follows.
Doing this relieves us of needing to count braces to see when an entire JSON message was received. To parse the structure we can do the following:
- Look for the opening ‘[‘.
- Then process numeric values until a ‘,’ is found. White-space is ignored.
- White-space is again ignored and N bytes are read.
- Finally remaining white-space is ignored and the trailing ‘]’ confirmed.
If the numeric value is invalid, the comma missing or the trailing ‘]’ bracket not found, the entire message is to be ignored. The extracted JSON Object can then be validated as well.
- Fixed a bug that stopped the serial port from being read if an Ethernet connection was disconnected. This appears to be a bug that was introduced in version 5.3.
- Moved the serial configuration to take advantage of the AUXSerial and COMSerial settings that JANOS uses.
Release Date: May 10 2019
MODBUS is a communications protocol used to communicate between a master and a slave or several slaves. The JNIOR implements MODBUS TCP. MODBUS TCP is the form for this protocol over the Ethernet network. The JNIOR acts as a slave by accepting requests and forming responses. Since Ethernet networks call devices clients or servers, the JNIOR is a server. Therefore the JNIOR has an application called MODBUS Server to handle the MODBUS TCP requests.
The MODBUS Server application is NOT enabled by default. It must be enabled before MODBUS masters or clients can send requests to the JNIOR. Once enabled the JNIOR will begin listening to TCP connections on port 502.
To enable the MODBUS Server you should go to the DCP (Dynamic Configuration Page).
Then navigate to the ‘Configuration’ tab.
Now, select ‘Applications’ halfway down the left side. Make sure the MODBUS Server application is checked
Once the application is checked it is then required you Reboot the JNIOR.
You can then optionally make a telnet connection to verify. The ps command to make sure the modbusserver application is running. Then you can use the netstat command to see that port 502 is listening.
Many times the best way to send diagnostic information to INTEG is to send us a Snapshot. A Snapshot is a zip file containing all of the files from the JNIOR.
When a Snapshot is taken the Support Tool will make a telnet connection, run a few diagnostic commands and then download all of the files. The diagnostic commands that are executed will be logged to the snapshot.log file.
There are two commands that will generate files that are not normally in the file system.
The first in the iolog command. The iolog command will generate a jniorio.log file. This file shows the past 512 digital input transitions as well as the past 512 relay output transitions. Up to 16 relay outputs will be shown in the log. The relays on the expansion modules will be logged as well.
The second command is the netstat -c command. This command generates a network capture file in the temp directory. The network capture file can be loaded in a desktop application called wireshark. This file will grow up to 512Kb. The duration of time covered by the file depends on the amount of traffic that the JNIOR handles. Optionally filters can be applied to the capture buffer to limit the types of traffic that are logged.
To take a snapshot follow these steps:
- Open the Support Tool. Go to the Snapshot tab.
- Click on ‘Take Snapshot’.
- Answer ‘Yes’ to the Snapshot confirmation dialog
- Select the JNIOR you want to take a Snapshot from and click ‘Ok’.
- Once your snapshot is compete find it on the left. It should be made visible once it is complete. Right click on the Snapshot file and select ‘Show in folder’. You could also expand the zip archive in the tree view or double click it to open in your loaded Archive application.
It’s been 5 months since the release of 1.7.1. We finally had a reason to touch JANOS. We implemented the JMP protocol. JMP (JANOS Management Protocol) is the same JSON messaging structure that is used in our weboscket protocol.
The JNIOR Control Panel is shipped with a ‘click’ sound that plays when a switch is pressed. This gives immediate feedback to the user that the panel has power and that the software is running normally and is ready to accept the press. There are times when the panel is used in a situation where the ‘click’ may not want to be heard. In this situation we may want to turn off that sound. The following application will accomplish this goal.
To run this application you need to download it from the button and load it on your JNIOR. Applications are generally loaded in the flash directory. To run it simply type the command as you see it above.
ControlPanel Application JAR [ Mar 27 2019, 7.43 KB, MD5: c573b4c989ea0380729fecae5283e937 ]
In the following code you will notice two classes. The main class of interest here is the ControlPanel class. This class will load the arguments into a ParameterGroup class so that we can process different commands at some point as well as there specific options. For now our command line is pretty simple but we set this application up for the future.
package com.integ;
import com.integpg.sensor.SensorPort;
import com.integpg.system.JANOS;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.Hashtable;
/**
* This application will allow you to set the volume of the control panel click
*/
public class ControlPanel {
private static final String USAGE_STRING = "CONTROLPANEL\n\n"
+ "Options:\r\n"
+ " -VOLUME volume Sets the volume level to a value between 0% and 100%\n"
+ "\n"
+ "Set options for the control panel";
public static void main(String[] args) {
// load the arguments into our parameters class
ParameterGroups parameters = new ParameterGroups(args);
// if the paremters are empty or there is a -help parameter then print the usage string and exit
if (parameters.isEmpty() || parameters.containsParameterGroup("help")) {
System.out.println(USAGE_STRING);
return;
}
// go through each parameter and process it
while (parameters.hasMoreParameters()) {
String parameterGroupName = parameters.getNextParameterGroup();
ArrayList<String> parameterOptions = parameters.getParameterOptions(parameterGroupName);
switch (parameterGroupName) {
case "volume":
try {
// process set volume
processSetVolume(parameterOptions);
} catch (Exception ex) {
System.out.println(ex.getMessage());
}
break;
default:
System.out.println("unknown parameter: " + parameterGroupName);
return;
}
}
}
private static void processSetVolume(ArrayList<String> options) {
// ArrayList<String> volumeArguments = _parameters.get("volume");
if (0 == options.size())
throw new RuntimeException("must specify a volume parameter between 0% and 100%.");
int volume;
try {
// get the volume from the arguments. allow the user to enter a double but
// save the integer part
volume = Double.valueOf((String)options.get(0)).intValue();
if (0 > volume || 100 < volume) throw new Exception();
} catch (Exception ex) {
throw new RuntimeException("invalid volume specified: '" + options.get(0)
+ "', volume parameter must be between 0% and 100%.");
}
// go through the external devices and set the volume for any connected control panels
for (long externalModuleAddress : getExternalAddresses()) {
// get a string representation of the address
String addressString = getAddressString(externalModuleAddress).toUpperCase();
// determine the type of module and handle accordingly
if (0xfa == (externalModuleAddress & 0xff)) {
try {
byte[] writeBlock = getSetVolumeWriteBlock((int) (volume * 2.55));
SensorPort.writeDeviceBlock(externalModuleAddress, writeBlock);
// lets print the result to the screen as well as the syslog
String result = String.format("Panel at %s set volume to %d percent", addressString, volume);
System.out.println(result);
JANOS.syslog(result);
} catch (IOException ex) {
throw new RuntimeException(
String.format("unable to write to the control panel: %s.", addressString));
}
}
}
}
private static long[] getExternalAddresses() {
try {
// query the unit for an array of external module addresses
return SensorPort.externalDeviceList();
} catch (IOException ex) {
throw new RuntimeException("unable to get external device list.");
}
}
private static String getAddressString(long address) {
String addressString = "0000000000000000" + Long.toHexString(address);
return addressString.substring(addressString.length() - 16);
}
private static byte[] getSetVolumeWriteBlock(int volume) {
// the write block for the control panel is 20 bytes
byte[] bytes = new byte[20];
// to set the volume we only need to set the command byte and the audio volume byte
bytes[10] = 4; // set audio volume command
bytes[11] = (byte) volume; // audio volume level
return bytes;
}
}
/**
* A class for interpreting command line arguments into parameter groups
*/
class ParameterGroups {
private final Hashtable<String, ArrayList<String>> _parameters = new Hashtable<>();
private Enumeration<String> _enumeration;
public ParameterGroups(String[] args) {
// parse the arguments
ArrayList<String> options = null;
for (int i = 0; i < args.length; i++) {
String arg = args[i].toLowerCase();
if (arg.charAt(0) == '-') {
while (arg.charAt(0) == '-') {
arg = arg.substring(1);
}
options = new ArrayList<>();
_parameters.put(arg, options);
} else if (options != null) {
options.add(arg);
} else {
System.out.println("Illegal parameter usage");
return;
}
}
_enumeration = _parameters.keys();
}
public boolean isEmpty() {
return 0 == _parameters.size();
}
public boolean containsParameterGroup(String paramterGroup) {
return _parameters.containsKey(paramterGroup);
}
public String getNextParameterGroup() {
if (null == _enumeration) _enumeration = _parameters.keys();
if (_enumeration.hasMoreElements()) {
return _enumeration.nextElement();
} else {
return null;
}
}
ArrayList<String> getParameterOptions(String parameterGroupName) {
return _parameters.get(parameterGroupName);
}
boolean hasMoreParameters() {
return _enumeration.hasMoreElements();
}
}
Once the ParameterGroups class has been instantiated with our command line arguments we loop through each parameter group. When we find the “volume” parameter group we call processSetVolume() and pass in the parameter options.
// go through each parameter group and process it
while (parameters.hasMoreParameters()) {
String parameterGroupName = parameters.getNextParameterGroup();
ArrayList<String> parameterOptions = parameters.getParameterOptions(parameterGroupName);
switch (parameterGroupName) {
case "volume":
try {
// process set volume
processSetVolume(parameterOptions);
} catch (Exception ex) {
System.out.println(ex.getMessage());
}
break;
default:
System.out.println("unknown parameter: " + parameterGroupName);
return;
}
}Now that we are inside the processSetVolume() method we need to validate the options provided. We make sure that a volume was provided and that it is a valid number between 0% and 100%.
private static void processSetVolume(ArrayList<String> options) {
// ArrayList<String> volumeArguments = _parameters.get("volume");
if (0 == options.size())
throw new RuntimeException("must specify a volume parameter between 0% and 100%.");
int volume;
try {
// get the volume from the arguments. allow the user to enter a double but
// save the integer part
volume = Double.valueOf(options.get(0)).intValue();
if (0 > volume || 100 < volume) throw new Exception();
} catch (Exception ex) {
throw new RuntimeException("invalid volume specified: '" + options.get(0)
+ "', volume parameter must be between 0% and 100%.");
}
...
Now that the options have been validated and we processed the assigned volume we are ready to write to the connected control panels. We loop through each connected panel and check to make sure it is a Control Panel which is type 0xFA. If it is a Control Panel we can get the write block and send it to the Sensor Port.
// go through the external devices and set the volume for any connected control panels
for (long externalModuleAddress : getExternalAddresses()) {
// get a string representation of the address
String addressString = getAddressString(externalModuleAddress).toUpperCase();
// determine the type of module and handle accordingly
if (0xfa == (externalModuleAddress & 0xff)) {
try {
byte[] writeBlock = getSetVolumeWriteBlock((int) (volume * 2.55));
SensorPort.writeDeviceBlock(externalModuleAddress, writeBlock);
// lets print the result to the screen as well as the syslog
String result = String.format("Panel at %s set volume to %d percent", addressString, volume);
System.out.println(result);
JANOS.syslog(result);
} catch (IOException ex) {
throw new RuntimeException(
String.format("unable to write to the control panel: %s.", addressString));
}
}
}| Name | Version | Release Date | Size | MD5 |
|---|---|---|---|---|
| MODBUS to MQTT | v1.2 | Mar 26 2019 | 562.5 KB | 791e3bbe95d6da1d66493e612e8f4869 |
| MQTT | v4.3 | May 28 2024 | 557.1 KB | 334c92d987049c220dd7defbfedc55ae |
Got a MODBUS device that you want to publish its data to the cloud? Use the JNIOR and the MODBUS to MQTT application.
MODBUS is a protocol that many legacy systems use. MQTT is new and works very well to publish data to any MQTT broker whether in the cloud or hosted locally. The JNIOR can fill the gap and get that older equipment’s data from the equipment to where you need it for analysis.
Set up the MQTT broker
There are several free brokers that you can use to test with. You can find that list here: https://github.com/mqtt/mqtt.github.io/wiki/public_brokers
Define your slaves and the tags
If your slave is RS485 then you will need to use the model 410 and set up the serial settings to match your device.
Once your setup is complete you will the the data update on the MODBUS to MQTT page. A message will be sent around the message pump the the MQTT application to publish the data. The data will be sent to the topic as follows with spaces in the slave name or data tag simply removed.
This application is still a work in progress so please register to leave a comment below!
Sometimes a JNIOR gets re-purposed. A JNIOR was performing one task and now it is going to perform a new one. You may want to clean up the JNIOR before configuring it for the new operation. In this case we will want to sanitize the JNIOR. To complete this task there will be two steps that will need to be performed.
First, we will need to “erase” the unit. To do this we will execute the following command;
reboot -eraseall
After confirmation the JNIOR will erase itself. It will be completely blank. Only the operating system will be present. All other software will be removed. The title says reset to factory but the OS will still be the currently loaded version and will not revert to the version that was shipped. The IP Address will also remain unaffected. All other user configuration including the hostname will be cleared.
That leads into the second step. You will need to get the software package that you wish to install and update the unit with that code. Use the support tool and the Update tab to do this.
After doing a factory reset, we recommend updating a JNIOR with the latest JANOS All-In-One update project. Updating this to your JNIOR will restore Web UI functionality for configuring your JNIOR, as well as return applications that came with the JNIOR when it was built.
What is MQTT?
This is what MQTT.org says:
MQTT is a machine-to-machine (M2M)/”Internet of Things” connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium.
Here is a great intro to MQTT. NOTE: This application is in development and screenshot may appear different when released. You can download the latest release here.
MQTT v4.3 [ May 28 2024, 557.11 KB, MD5: 334c92d987049c220dd7defbfedc55ae ]
Use MQTT with the JNIOR
Currently the MQTT functionality is limited but we are always looking for suggestions. Currently will publish IO status by default. When the JNIOR boots up and successfully connects to a broker we will report the current status of the digital IO. Digital inputs will report state, usage meter and counter values. Relay outputs will report just state and usage meter values.
To successfully connect to a broker you will need to have selected a broker and entered its information. Information such as the Host URL, port number and whether the connection should be encrypted. The JNIOR implements MQTT across a TCP connection. The default port number for MQTT is 1883 when non-secure connection while it is 8883 when using a secured connection. There are several public test brokers out there. Do not use any free broker for sensitive information. Brokers that require sign up will most likely provide a user name and password.
Groups!
There are two types of groups, Device Groups and Relay Output Groups. Both of these concepts are in BETA as of version 1.2 and we are looking for feedback.
Device Groups allow JNIORs to subscribe to topics that contain the Device Group name instead of the Serial Number. A JNIOR can be part of one or many Device Groups. By default a JNIOR will subscribe to topics that are meant for it by Serial Number, for instance, jnior/618010001/set/... The JNIOR may want to be part of the pump stations group for example the JNIOR will subscribe to jnior/pumpstations/set/... topics.
Relay Output Groups allow the specification of more than one relay to be affected at once. To set relay channel 1 you can publish the jnior/SERIAL_NUMBER/set/digital/outputs/1/state = true. But maybe you want channels 1 – 3 to be affected at once. To do this you will define a named Relay Output Group, “lights”, for example. Then, when the JNIOR receives the jnior/SERIAL_NUMBER/set/digital/outputs/lights/state topic all defined relays will change at once.
Enhance the MQTT implementation!
You can enhance the MQTT implementation on the JNIOR with a custom application. Any topic that the JNIOR receives that is not part of our native implementation will be posted to the message pump for the consumption by another application. A custom application may also put topics with payload data to the message pump to be published by the MQTT application. This way the MQTT application performs the protocol logic while the custom application can worry about the logic.
- Watchdog no longer causes reboot from foreground execution
- REGEX Alternation ‘|’ has been corrected to properly work in Group
- Corrected NULL Pointer write issue when a Group which employs alternation appears at the start of a REGEX expression
- Resolved message pump memory issue relating to process termination
- Expand the channel range for setOutputRelay() from 0-11 to 0-15
- Corrected network capture filtering when an IP address is to be excluded
- Supports networks using Jumbo Frames
- Eliminated assertion associated with improper use of ‘%n’ in
System.out.printf.
JANOS 1.7 Released July 26, 2018
- Added ability to load single CA Certificate to be supplied on TLS connection
- Support TLS Client Certificate Verification on outgoing connections
- Added legacy PKCS1 support for externally generated certificates
- Corrected array issue with processing of deferred email transmissions
- Corrected memory issue with TAB use on the Command Line
The JniorWebSocket Library was started as an example project. It has not had much use here in-house and therefore the testing has been limited.
Here is a quick application that demonstrates how to instantiate multiple JNIOR WebSocket objects. Each Object is added to a tree control so that they may be interacted with individually. So far I have added the ability to add and remove, connect and disconnect and pulse output 1. This application can be easily extended to manage multiple JNIORs and perform many different tasks.
Here are 2 links. The first is a link to the Visual Studio project for the library. The second link is a zip file containing the example described above.
C# WebSocket Example [ Oct 30 2018, 6.24 MB, MD5: fe1be1e450c2772a16d090a0d9c37f99 ]
MultipleJniors-1.zip [ Oct 31 2018, 1.05 MB, MD5: 26485d581f00d9c5985db3e7f481e551 ]
The key to debugging these days lies in the ability to recreate the failure. Some of the more elusive problems start out as rumors where a user swears that something failed to do what was expected but that he/she cannot prove it. After a while perhaps the issue resurfaces and this time someone is able to collect enough evidence to barely prove that the problem is a real concern. What exactly occurs remains a complete mystery. Eventually someone files a bug report but the developers are clueless and cannot effectively address the cause.
Sometimes there are suggestions as to what actions may have triggered the problem. The Support Team may take the time to try to recreate the problem on a local system but to no avail. Other issues which stand a better chance of being resolved take precedence. Meanwhile news comes in that the original issue has again occurred and may even now happen in completely different locations and environments. What to do next?
Perhaps it is appropriate to regenerate the application for the user’s site to add logging and other debugging statements in hopes of creating a better record of the events leading up to a future failure. The difficulty here is in deciding what exactly to log and to be logged by what aspects of the system given no real knowledge of the source of the issue. Some guesses are made and the logging is enabled. Some time later which could be weeks the issue reoccurs and the logging proves to have not been useful. It may even be that the content of the logs are now meaningless as the logic behind them has been forgotten. So it is difficult to make progress.
It would be nice to know when the issue is going to occur so you know exactly when you need to start watching for it. Better yet, if the problem occurs you could skip back in time to only a few seconds before and you could solve it! If the issue involves network interaction you could setup a sniffer like Wireshark and set it to collecting detailed network traffic. Now with an external device this may involve the use of a Network Hub as opposed to a Network Switch so you can add a machine running Wireshark that would see all of the traffic. But that is a lot of hardware and it might just not be feasible.
Here’s where JANOS and the JNIOR might surprise you. If the issue is with the JNIOR application you can use the netstat -c command from the Command Line interface to generate a PCAP capture file capturing network traffic. That PCAP file can be downloaded to a PC with Wireshark installed and it will open right up for analysis. And here’s where the magic occurs. It is almost like JANOS knows ahead of time when the issue will occur, that PCAP file can contain traffic from BEFORE the event happened although it has been generated after the fact.
Okay. So it is not magic but the foresight of the operating system which always is capturing network traffic. There is a capture buffer that by default is 512KB and that can be enlarged through the IpConfig/CaptureBuffer Registry Key to as much as 8MB. It is not so much a ‘buffer’ as it is a queue. At any one point in time it contains a significant amount of recent network traffic. You can also set filtering that is to be employed during capture to limit the collected traffic to that of concern and thereby greatly extend the time frames involved. You can also use a filter in generating the PCAP file to limit what you transfer for analysis.
Now what if the event occurs overnight and by the time you realize it the data that you need has been overwritten? A simple utility application can be added to the unit to automatically generate the network PCAP capture file when the event is detected. You can even then have that file emailed to you!
Those of you who may have worked with this aspect of the JNIOR will note that the network capture only includes data from the previous boot if it is not yet begun to wrap around. If the failure involves an assertion that might cause the JNIOR to reboot the information is lost. Well this is true, however, as of JANOS v1.7.1 which now as of this writing a Release Candidate the capture buffer is non-volatile. It will survive a reboot and contain a record of the prior network traffic. The buffer is only reinitialized when power is applied.
It is also possible to configure the JNIOR in Promiscuous Mode wherein it would capture all external network traffic and not only that targeted to the JNIOR. If you couple that with a Network Hub you could utilize the JNIOR to monitor the interaction of other devices on a local network segment. I bet you never thought of JNIOR as a IT debugging tool?
If this aspect of the JNIOR interests you or if you need to utilize it in debugging, please feel free to contact INTEG for support. You may also consult the Registry Key Assignments documentation for details on network capture. This information is supplied as part of the Dynamic Configuration Pages (DCP) served by the JNIOR WebServer accessible by your favorite browser.
In an earlier article we attempted to connect the JNIOR to a DMX512 network in a way that would allow a theater stage crew to control relays and other outputs through their main lighting control panel. Here with the programmable facet of the JNIOR we could help create complex special effects that can be triggered in-sync with lighting. In fact, DMX channel data can be used by the JNIOR application to trigger and manipulate activity on a LAN segment and thereby update computer screens and TV displays on stage. This can all occur with precise timing in concert with other lighting changes.
The DMX512 protocol and interface was developed over 50 years ago as a means to reduce wiring cost and to improve the mobility needed to support major concert tours in the music industry. It incorporated serial data transmission formats that were state-of-the-art at the time and that are still relevant today. Unfortunately the protocol employed a baud rate much higher than would be later recognized as standard in the computer industry. Also the limited capabilities for data synchronization and lack of error checking make interfacing difficult. Even though the JNIOR Model 410 is able to handle the RS-485 signals and was upgraded to receive 250 Kbaud data triggered on the specified serial Break Condition, we could not achieve reliable operation as a fixture through programming alone.
The issue relates to the design of the standard computer hardware component called the UART. The Break Condition that the DMX protocol uses to identify the start of a frame of data is not handled by the UART in a fashion that can be reliably used to synchronize data reception. At least it is not possible in a purely software implementation as we discussed previously. The answer lies in a simple hardware adapter that alters the DMX512 signals just enough to work correctly through the UART. We will see what that entails in the rest of this article.
DMX512 Fundamentals
Today DMX is defined by official standards. On the wire DMX512 uses EIA-485 differential signaling. We often refer to it as RS-485. You are probably more familiar with the term RS-232. As serial digital communications became more prevalent the distance limitations and noise susceptibility of RS-232 became more and more of concern. The solution, designated as RS-422, used balanced transmission lines over twisted pair which extended the distances that signals can traverse as well as the communication rates that can reliably be achieved. RS-422 connections however are point to point and as computer systems grew so did the need for networking. Soon RS-485 came along and added functionality that created a multi-drop environment where one talker can communicate with a number of listeners. RS-485 is what DMX512 needed where one (or more) lighting control panel must communicate with multiple lighting fixtures.
This basically amounts to a 3-wire network where there is one DATA+ (positive) data line and one DATA- (negative) data line along with a GND ground. The DMX standard details specific connectors (5-pin XLR) and wiring of sufficient gauge and quality. The world also wants to utilize 3-pin XLR connectors for DMX as these are prevalent in the industry through their use in audio applications. You will find a mixture of fixtures some with 3-pin and some with 5-pin connectors. Thankfully 3-pin to 5-pin adapters are readily available.
On the protocol side it was necessary to standardize. This would encourage lighting manufacturers to incorporate the technology. Over the years the protocol itself had been handled by different standardization groups and now is:
American National Standard
ANSI E1.11-2008 (R2008)
Entertainment Technology – USITT DMX512-A
Asynchronous Serial Digital Data Transmission Standard
for Controlling Lighting Equipment and Accessories.
In order to understand where we have difficulty we need to look deeper into the specifics of what is sent across the wires. You recall that DMX was invented to reduce wiring costs. Each channel in the protocol replaces a single power cable. The protocol allows for up to 512 channels and all of those multiplexed onto a single 3-wire network cable. That is a significant savings.
Each channel digitizes an analog signal into one 8-bit value. These can represent numbers from 0 to 255 and for lamps that is used to sufficiently cover the range from 0% to 100% brightness in 256 steps. Where once one channel controlled one lighting fixture, today multiple channels are assigned to each fixture to define attributes such as color and positioning in the case of motorized fixtures. In the serial world each channel is transmitted just as a byte of character data would with a start bit, 8 data bits and one or two stop bits. For DMX512 two stop bits are used. That is a total of 11 bits per channel.
The DMX protocol allows for the transmission of up to 512 channels. These are sent in sequence starting with channel 1 and up to channel 512. A Start Code is defined as channel 0 and the typical value for that is zero. The complete set of channels 0 through 512 (513 channels) is called a Frame. Frames are transmitted repeatedly one right after another with the start bit immediately following the prior stop bits at the defined rate of 250 Kbaud. We can now do some math. With 11 bits per channel and 513 channels that is a total of 5,643 bits. A bit at 250 Kbaud requires 4 microseconds and so the data part of the frame takes a total of 22.6 milliseconds. We can fit a little over 44 frames in a each second.
Now with a constant flow of channels each looking the same on the wire how do we know which one is channel 0 and which might be the one we need for our fixture? If we connect to an active DMX network we need a way to synchronize ourselves. For this the standard defines the use of a Break Condition to signal the beginning of a frame. In old days this was called an Attention Signal and it was used to wake up remote teletype equipment and there was actually a key on the Teletype for this. So back in those days this was a natural choice for DMX synchronization. A Break Condition amounts to a period of time where the signalling is held at the level of a Start Bit (Space) long enough to violate the normal byte format and cause a reception error. The receiver simply fails to see the Stop Bits where they are expected and raises a flag. This is intended to signal the beginning of a frame.
By this definition a Break Condition need only last long enough to mask the expected Stop Bits (11 bits, 44 microseconds). This causes a UART to signal a Framing Error indicating that the data it received was not properly formatted and followed by the expected Stop Bits. This error can then be used for Break detection and the synchronization of the DMX Frame. The current DMX512 specification defines the minimum time for this Break Condition to be 88 microseconds which is exactly the transmission time of two channels. The timing of this Break Condition has been reduced from prior versions of DMX512 and it is unclear if there is any risk that existing lighting fixtures may fail to operate when presented with the shorter pulse. The specification does not specify a maximum duration for the Break Condition and states only a “typical” duration of 176 microseconds.
After the Break Condition terminates there must be a brief period of time before the Channel 0 (Start Code 0x00) can be transmitted. This period is called Mark After Break and the current specification defines a minimum of 8 microseconds the time period of two bits. This too has been reduced from prior versions of the specification. A maximum duration of 1 second is defined for Mark After Break and no typical value is given.
Here is what this looks like in reality. The wider pulse in the center is a Break Condition and Frame data can be seen to the right and left of it. Note that in this case all Channels are set to 0x00 and so we see only the Stop Bits for each.
One side note is that shown above are the CMOS logic signals observed after the RS-485 receivers. This signal is typically fed to the receiving hardware or UART. This image is consistent with our prior diagrams where the Marking condition is a HIGH or 1 and the Spacing condition a LOW or 0. The signals on the actual wire demonstrate the same timing but would involve dramatically different voltage levels.
There is one point that I have not mentioned. The total number of Channels included in a Frame is a maximum of 513 but that number is optional. A lighting controller may decide to only send 128 Channels in addition to the START CODE assuming that others are unused and thereby increasing the number of Frames per second. The specification defines a maximum Frame rate of around 836 Hz (Frames per second). This can accommodate fixtures that need higher Refresh Rates. The Refresh Rate for a DMX512 Universe of a full 512 Channels is around 44 Hz.
In the previous article we alluded to there being some difficulty in receiving these signals with standard computer hardware. We now have enough technical detail to understand what that reception problem might be. This is the topic for the next section.
The Issue with the DMX Break Condition
Now let’s look at the task of receiving a frame of DMX data from a software point of view. We will assume that you have configured the hardware to receive RS-485 signals. The JNIOR Model 410 AUX port has RS-485 capability when properly wired and configured. INTEG can provide those details and those are mentioned in other articles. The JANOS operating system has also been enhanced to accommodate DMX and includes the 250 Kbaud rate setting as standard.
In the previous article JNIOR as a DMX Fixture we covered both the proper hardware configuration for the JNIOR Model 410 and the simple Java application software that might be used to receive a frame of channel data. We took that a step further and let the JNIOR activate relays in response to changing channel levels. It is here that we noticed a reliability issue and then spent time to understand it. We discovered that it is a limitation in the design of the standard UART and not something specific to our software or even the design of the JNIOR. Here we will review those findings and then in the balance of this article present a solution.
As you can imagine it would not be acceptable for the JNIOR to receive an incorrect channel value and act upon it. On stage you might create a flicker and distract the audience or worse trigger some elaborate effect at a totally inappropriate moment. Neither contribute to receiving good performance reviews. Well our first attempt at having the Model 410 serve as a fixture worked but occasionally there was a glitch. A relay toggled inappropriately and this lead to an investigation.
We started by attempting to validate the frame of channels. We know that the START CODE should be 0x00. So we decided to first test the START CODE. Well, 0x00 was not what we were seeing and this led us to understand that the UART is not able to properly receive data immediately following a Break Condition. Let’s look deeper into it.
Every description of the inner workings of the UART describes the reception of data beginning with the detection of the Start Bit. They state that “A Start Bit is detected as the falling edge of the signal” (which at that point is moving from a Mark condition to a Space condition from HIGH to LOW). This makes sense since this can be used to synchronize the baud rate and the UART once detecting the falling edge delays 1/2 bit time to begin sampling. The first reading is then LOW (0) since it is a start bit.
The UART then can proceed to receive data bits one after another starting with the LSB (Least Significant Bit or D0). Let’s assume that we are configured for 8 data bits and no parity as is the case for DMX. Once all 8 Data Bits are collected and the byte of data fully accumulated then next bit sampled is expected to be HIGH (1) as this is the necessary Stop Bit. The UART sees the Stop Bit and pushes the data to the output buffer and notifies the system that data can be read from the port. You can see how this follows from the first figure in this article repeated here.
Logically then you can see how the UART might go back into some ‘Start Bit Search Mode‘ in order to pass over any number of remaining Stop Bits and dead time between character transmissions.
We mentioned previously that in the case of a Break Condition the Stop Bits are missing. Here the UART receives 8 Data Bits or what it hopes are 8 Data Bits and then fails to detect the Mark condition signalling the presence of a Stop Bit. The typical UART then pushes the data to the output buffer and signals the processor that data can be read from the port. It also sets a Framing Error flag and well-written serial drivers handle the error accordingly.
Here is where your logical thinking might fail you. Once the UART signals a framing error you would like to think that it reenters the aforementioned ‘Start Bit Search Mode’. This would take the UART through the balance of the Break Condition and past whatever Mark After Break is present to the very next falling edge of the signal. This then allowing it to reliably and properly receive the very next valid byte of data. Well, surprise, it does not work that way.
Instead the UART continues to sample bits based upon its internal baud rate clock. It accepts the very next LOW (0) bit as a Start Bit and proceeds to process the following 10 or 11 bit times as another byte of data. This is why from the software side a Break Condition usually presents itself as more than one 0x00 byte each with an associated Framing Error. By itself this is not an issue as you can accept one or more Framing Errors as an indication of the Break Condition. In fact you could use this to detect a short Break from a long Break. But let’s look closer at what happens at the very end of the Break.
At the conclusion of an arbitrarily long Break Condition the signal returns to a Marking state. We have reached the Mark After Break but it is highly likely that the UART is still accumulating hopeful data bytes. Eventually it sees the Mark as a valid Stop Bit. The result is that a byte is then output by the port and there is no Framing Error. There is no way for software to determine if this is the first valid data byte (START CODE) or a bogus trailing byte formed out of the end of the Break Condition.
In testing we can see that this indeed is occurring. Since data bits are transmitted from least significant to the most significant, depending on signal timing this bogus byte can be seen to contain any one of the following: 0x00, 0x80, 0xC0, 0xE0, 0xF0, 0xF8, 0xFC, 0xFE or 0xFF. Perhaps you can see that this result depends on just what Data Bit the UART thinks it is receiving at the point when the Mark After Break begins. There is actually one additional case and that being when the data is actually the first valid data byte after the Break. This effect depends highly on the duration of the Mark After Break as well.
Matters can be even worse. In DMX512 the Mark After Break can be as short as two bit times or 8 microseconds. That means that it is possible that the UART does not even look for a Stop Bit until the signal is deep into the START CODE byte. Since the START CODE is itself 0x00 the UART might then grab one of those data bits as a Start Bit and, well, fail to receive several channels of data. How can this be? Are UART designs defective?
Well let’s not jump to that conclusion just yet. Let’s give the hardware developers the benefit of the doubt at least initially. First we should understand that once the computer industry embraced serial data communications use of the Break Condition fell by the wayside. Protocols used Start of Header (SOH) bytes and other means for synchronizing data blocks. These tended to include byte counts, error checking and even error correction. These days there is even data compression and data encryption. The Break Condition not only fell out of use but also started to disappear from serial communications documentation.
Meanwhile the industry started to look into the reliability of data transmission lines. In the early days RS-232 cabling was not what it is today and it was very susceptible to external electrical noise. Also remember that we were still using machines developed in the early half of the 20th century. These were power hungry electro-mechanical contrivances that spewed out electrical noise with reckless abandon. RS-232 signals often experienced data disruptions some as brief as an incorrect value of a single bit up to the loss of entire blocks of bytes. There were even situations where communications were completely blocked out for lengthy periods of time. Perhaps for as long as the nearby noisy equipment remained in use. This led to better cabling. It led to improved communications standards like RS-422. It prompted the need for error detection and motivated a lot of creativity on the error correction front. Simultaneously the Federal Communication Commission (FCC) put forth standards for controlling RF and electrical noise emissions and prompted certifications making new equipment meet the stringent requirements.
A UART design that utilized a “Start Bit Search Mode” fails to perform properly in a noisy environment. In the case when a Stop Bit is damaged by noise and thereby missed by the UART, an approach that accepts the very next falling edge as a start bit falls on its face. It fails to re-synchronize quickly. Many bytes of data then are lost even though electrically the signals are undamaged. Error correcting schemes just could not handle it. They determined that the current UART approach performed much better and, well, what is this Break Condition anyway?
We can only guess at how we got to where we are today. It is actually a surprise that a 50-year old protocol would still be as active today and it would still rely on synchronization techniques like the Break Condition. Of course it is not really an issue as DMX fixtures are developed to properly receive these signals. We are just a bit hampered in trying to use standard computer hardware. So what can we do about it? That is our next topic.
Correcting the DMX512 Protocol
We stand little chance of “correcting” the DMX512 protocol. With over 50 years worth of legacy DMX equipment out there a change in the design of the protocol is just not an option. The ability for DMX to inter-operate with standard computer hardware is just not that critical of a need. We have also exhausted any possible software solution to this issue. So what can we do? Some kind of hardware solution will be needed if we want to develop a JNIOR that can be used as a DMX fixture. What form will that take?
Obviously lighting fixtures all over the world read DMX channels seemingly without difficulty. So we know we can do it. Does this require a custom UART implementation in some FPGA component? Do we have to dedicate some PIC processor or other device to process incoming DMX bit by bit? Are there standard DMX chip level devices out there to make it easy? It certainly all cries out for some research. But first let’s see if there is some simple and creative way to address the issue.
The problem breaks down into two separate issues:
- The UART is unable to synchronize after a Break condition and reliably read the first START CODE byte.
- The trailing edge of the Break Condition can generate unexpected data not identified with a Framing Error.
Clearly if we use a Framing Error indication to signal the reception of a Break Condition and we are 100% assured that the next valid data byte is accurately representing the START CODE we are good to go. We assume then that the processor has enough horsepower to receive and buffer all 513 channels (if they are present) without error. This can be done with low-level interrupt routines avoiding any dependence on the processing load in the JNIOR. In fact all of this has already been implemented in JANOS and deployed into the field. Those techniques were tested in the prior article.
The Mark After Break condition specified by the DMX512 protocol is a minimum of 8 microseconds. That is just two bit times. This is unfortunate. If this Mark After Break condition were to be held at least 11 bit times (44 microseconds) then any UART would be assured of seeing some part of the condition as valid Stop Bits. The UART then would be guaranteed ready to receive the very next data byte which is the START CODE.
If we somehow could extend the Mark After Break that would insure the reliable reception of the channel data. That by itself is not enough since we know that the trailing edge of the Break Condition can generate a data byte that is not flagged with a Framing Error. This would confuse us as there is no real way to determine if this is the START CODE or a bogus addition to the stream.
Now if we could somehow limit the Break Condition to a single byte we would receive one and only one byte flagged with a Framing Error. The UART would not see an additional Start Bit and thereby insure that the next byte we see is in fact the START CODE. We know that the Break Condition is at least 88 microseconds at its shortest. That is exactly two data byte times (each of 11 bits = 1 start bit + 8 data bits + 2 stop bits). So if we truncated the Break Condition at exactly 44 microseconds which would generate 1 byte with a Framing Error, there is another whole byte’s worth of time that could be added to the Mark After Break. This would satisfy our interest in extending the Mark After Break for at least a byte time. At a minimum it would then be 52 microseconds (44 borrowed from the Break + 8 original microseconds).
The Hardware Solution
Our hardware needs to truncate the Break Condition. We need only hold it long enough to mask the first Stop Bit and thereby force the Framing Error. This needs to be achieved without risk of causing other Framing Errors. With a little thought we developed the following logic.
The CD4024B (U6) is a 7-stage counter and it is reset (RST) or restarted with any HIGH Marking condition on the incoming data. This means that the count is not allowed to progress very far except during the Break Condition where there is no reset (RST). This in essence times the duration of the Break Condition. Note that when the output of the 7th stage goes high (count >= 64) the OR gate then pulls the TXD signal high effectively truncating the Break Condition. The signal output is then held HIGH until the Mark After Break arrives and resets the counter. At this point the incoming signal is HIGH and we have effectively transferred time from the Break Condition to the Mark After Break. This is exactly what we want.
The UART will throw the Framing Error as soon as the first Stop Bit is not detected. We don’t want it to see any potential Start Bit after that. To be safe then we truncate the Break Condition after the 10th bit time right after the first Stop Bit. This means that we must truncate the Break Condition precisely after 40 microseconds and this must occur at the 64th count when the 7th stage goes high. As it turns out 40 microseconds divided by 64 is 0.625 microseconds the precise period of a 1.600 MHz clock. Those are readily available. So we clock the counter with a 1.600 MHz signal using oscillator U4.
One last detail to consider. There is no upper bound on the duration of the Break Condition. That means that once we truncate the Break Condition we need to hold it for potentially a very long time. So we cannot allow the counter to continue to run. Once the 7th stage goes HIGH we need to hold it there. A second OR gate (U5) masks the incoming 1.600 MHz clock signal at that point effectively stopping the counting. And, miraculously, we’ve got it covered.
To test this we created a prototype adapter which implements an isolated DMX512 standard port. The signal passes through our circuit and then a standard RS-232 transceiver is used to communicate with the JNIOR AUX port. One advantage this has is that any Model JNIOR with an AUX port can be used as a DMX fixture. Whereas only the Model 410 supports RS-485 directly and can be used as described in the prior article.
Here the few components involved sit in the lower lefthand corner. These Surface Mount devices take up almost no real estate and this is perfect for a new model of the JNIOR sporting a DMX512 input. Okay, so does the circuit work?
Here we see another oscilloscope trace. The yellow signal is RXD as received from the RS-485 receiver. This is what enters the schematic from the left. The blue signal is TXD and what we will supply to the UART. This exits on the right of our schematic. The wider pulse in the center is our Break Condition.
On the yellow trace we see that the Break Condition supplied by the DMX network is about 100 microseconds and it is followed by a brief Mark After Break which looks to be somewhat less than 50 microseconds. We know that this signal is problematic as it is from the same source used in the prior article. Note here that the START CODE and all channels happened to be 0x00. So those high pulses are Stop Bit pairs.
Now the blue trace is the result. The Break Condition has indeed been truncated. The vertical cursor lines are positioned to measure the duration of the new Break Condition. On the right we see the delta-t to be the precise 40 microseconds we had hoped for. We can also see that the balance of the time in the Break Condition has now been added to the Mark After Break.
Does this allow the JNIOR to be used as a DMX fixture? In fact it does and there is absolutely no glitch or reliability concern. Here is a short video showing relays responding to a DMX channels. A channel value greater than 127 is used to signal closure of a relay. The JNIOR’s 8 relays are mapped to consecutive channels.
Of course this would need to be evaluated with a wide range of DMX512 signal sources. The Java application on that JNIOR in the video is essentially that discussed in the previous article. It would be useful to reiterate its operation but that is beyond the scope of this article. If you should be interested in using a JNIOR as a DMX fixture just let us know. We can likely supply you with one of these prototype adapters and the application programming is open source from us. If you already have a JNIOR then your are almost there.
We should note that not all UARTs are created equal. There are likely UART implementations that do properly handle synchronization after the Break Condition. If you have one of those then you are golden. In the case of the JNIOR we were not so lucky. While the adapter was described above as prototype we do have several that we can supply. It is not clear that there would be any long term demand for this. A model of the JNIOR directly providing a DMX512 electrically isolated input is now feasible and would likely include the circuit as described here. Contact INTEG to find out more.
Update
We can provide the adapter (as shown) along with a power supply and instructions. The adapter can be used in any serial application and not just with the JNIOR however as discussed in the article you would need to accommodate the unique Baud rate and trap the framing error for synchronization. The document supplied includes an example application for the JNIOR and a schematic.
Note that straight-thru terminals are provided allowing you to tap either 3-wire or 5-wire DMX signals. You may wish to splice into a DMX extension cable (not provided) to give yourself cabling terminated by the desired (3 or 5 pin) DMX XLR connectors. An isolated DMX port is implemented and so your JNIOR or PC is not directly connected to the lighting system.
The above kit is INTEG P/N A00-00155 – DMX512 SERIAL PRE-PROCESSOR.
Once a JNIOR is up and running in an application it rarely needs attention. In fact many users forget that it is even there. It is however good practice to check the system logs once and a while. Fortunately there are a number of ways that you can do that. If you open the DCP (Dynamic Configuration Pages) by accessing the unit with a browser there is a ‘Syslog’ tab. This displays the log with the most recent events right at the top. You can also go to the ‘Console’ tab or otherwise open the Command Line through a Telnet connection or the serial port where you can list the log content directly to the screen or window. In this case the latest entries are the last listed.
If you are developing a new application and testing it on the JNIOR you may wish to monitor the log more closely. Naturally you can leave the DCP open on the ‘Syslog’ tab and new events would just pop up when they are logged. That might not be as convenient as it sounds especially if you are building a network-facing application and are constantly testing and working with the JNIOR using various browsers. Perhaps you are rebooting the unit causing the DCP to reconnect far too often. More typically we periodically make a Command Line connection through Telnet and manually list the log. Here we make use of the CAT command.
mqtt-test /> help cat CAT filespec Options: -H Dump file in hex -J Formats JSON -P Displays last page Displays the contents of a file. Aliases: CAT, TYPE mqtt-test /> cat -p jniorsys.log
Now the jniorsys.log file is limited to about 64KB characters. That can represent a lot of lines. Do you really want to list those every time? Here we see that the CAT command also provides the -P option which will show only the last page (25 lines) of the file. Of course you do have to remember to include the option. One side note here is that with JANOS you can supply the options anywhere on the line. So if you at first forget to include the -P you can place it after the filename. No problem.
We can improve on this by making use of JANOS batch file capabilities. A batch file is a kind of script file that can contain commands to be executed one after another. Most operating systems, JANOS included, provide some facility for batch execution. But you might be wondering how do you create batch files or if you do that externally how do you get them onto the JNIOR? Well here’s a tip: You can create them on the fly. Of course, the simpler the file the easier that can be accomplished.
For example we will create a simple batch file called log.bat as as kind of short hand that we can use when we want to display the System Log. Here we simply ECHO the command routing the text to the file. Immediately we can use the short batch file name to execute the command.
mqtt-test /> echo cat -p jniorsys.log > log.bat mqtt-test /> log cat -p jniorsys.log 10/03/18 18:54:15.850, FTP/10.0.0.27:58531 uploaded /flash/MQTT.jar [325.8 kbps] 10/03/18 18:54:24.571, ... (etc.)
Here we see that we used the ‘>’ pipe character to route the output to the specified file. If you need to build a batch file with more than one command you can append to it using ‘>>’ as well. The power here comes in combining commands to achieve some goal. The advantage being that you can repeat the procedure easily using the batch file name as shorthand. While batch files in JANOS are not as fully featured as you will find in Linux or MS-DOS systems most of the basics are there.
Suppose instead we want to list only today’s events. Here we create our batch file in way that lists only those entries with today’s date. Check it out.
mqtt-test /> echo grep %DATE% jniorsys.log > log.bat mqtt-test /> log grep 10/04/18 jniorsys.log 10/04/18 12:10:48.657, Ending session Command/10.0.0.27:64514 (pid 132) 10/04/18 12:37:53.642, Command/10.0.0.27:64075 login 'jnior' (pid 405) 10/04/18 12:38:05.179, FTP/10.0.0.27:64118 login 'jnior' 10/04/18 12:38:05.711 ... (etc.)
And there you go. You now have a simple way to check recent System Log activity. Now I bet that you are thinking how you might alter this to be even more helpful. Well, if you find that you cannot achieve what you have in mind just let us know. An advantage that you have with INTEG is a direct link to the technical team and the advantage we have is the power to implement what you need.
@integpg