Travel Trailer API

Introduction

I have recently purchased a travel trailer and desire to instrument and automate it. I want to control and monitor all of its systems using my iPhone or other mobile devices. When I am out in the sticks (defined by my son to be wherever there is no Internet connectivity) I want to connect to the trailer’s systems via a local WiFi network. When the trailer is in a location where LTE, or other data services are available, I want to interact with the trailer via the Internet.

While there are many systems to monitor and control, there are two fundamental technological pieces they all require. First, a WiFi network must be provided for mobile phones and other devices to connect. Second, a central controller is needed to manage the myriad of systems that will be connected to this network and manage the associated API.

Rather than boiling the ocean, I intend on focusing on one simple representative system, a weather station, and connecting it to our trailer. Success with this system requires solving many issues that will simplify the inclusion of additional systems in the future. The rest of this post will describe a path forward. A series of future posts will describe the details associated with each subsystem.

System Description

The goal of this project is to enable a mobile phone user to view weather data acquired from a weather station connected to our travel trailer. There are a number of modules or subsystems required to accomplish this:

  1. A mobile phone that connects to a WiFi network and an application that consumes an API that returns weather station data.
  2. A secure WiFi network or trailer area network (TAN). While security of weather station data may not be required, future projects will require it and we should design with security in mind.
  3. A central controller that connects to the provided TAN and exposes an API management tool. This will enable mobile phones and other devices to access available data in a secure and managed way.
  4. A 433 MHz receiver connected to a microcontroller based module will convert the signal acquired from the weather station into an appropriate API.
  5. A weather station that transmits collected data on the 433 MHz band.

The weather station will collect environmental data and transmit it over the 433 MHz radio frequency band. This signal will be acquired using a 433 MHz receiver that converts this analog signal to digital for consumption by a microcontroller based system. The digital signal will be parsed appropriately and exposed through an API. This API will be managed by the central API management system and exposed to other devices over trailer area network. The mobile phone will consume this API and an appropriate mobile phone app will present it to the user.

Chosen Components

There are many possible components to choose from to accomplish the goal and steps outlined above. The components I’ve chosen and describe below are simply representative of the myriad of possible choices.

Weather Station

I will use the AcuRite 5 in 1 Professional Weather Center model 01536. This product is compact, easily mounted on a trailer and broadcasts weather data on the 433 MHz band. The outdoor component measures the temperature, humidity, rainfall, wind speed and wind direction. This data is combined into a packet and broadcast periodically. The indoor component is a nice color display that displays this data and archives previously received data. The indoor and outdoor components are paired through a simple channel selection mechanism.

433 MHz Receiver

I acquired the Sunkee 433 MHz superheterodyne wireless receiver module to convert the analog signal broadcast from the weather station into a digital signal appropriate for consumption by a microcontroller based system. This module can operate from roughly 3 to 5 Volts and requires a simple wire antenna. It outputs a digital signal easily read by standard microcontrollers.

Microcontroller System

I have chosen to use the Corewind WiFiG25 microcontroller module. It has a 400 MHz ARM processor, plenty of memory, built in WiFi connectivity, numerous I/O lines of various types, runs the Linux operating system, supports many programming languages, and only costs about $30.

Software Components

With the receiver connected to the microcontroller module, the rest is up to the software. A simple program will be written to acquire the digital I/O data from the receiver and store it in memory. The Kong API management tool will be implemented on the microcontroller to expose the collected weather data via an API. This API can then be consumed over the WiFi network by a mobile phone and associated application and our task will be complete, for now.

Summary

The components described above, and a few others, will be combined to expose weather data via an API for consumption by mobile phones and potentially other devices. The next post will describe the setup and use of the receiver module and a description of the data collected from the weather station. A series of posts will follow describing the remaining steps to accomplishing our goal.

myDoorbell: Play that Tune

Introduction

In a previous post, I described the nine modules that make up myDoorbell. These are shown in the following figure. In this post I’ll address two modules that are difficult to describe separately, the microcontroller near the center of the figure and the MP3 decoder to its right.

Basic Operation

In my previous post I described a circuit that will translate a doorbell button press to a signal appropriate for consumption by the microcontroller. When this signal is received by the microcontroller, it takes two actions:

  • It raises a signal to the second microcontroller enabling it to raise external events. These events will be discussed in a future post.
  • It reads a MP3 encoded audio file from the memory and writes it to the MP3 Decoder module.

The MP3 Decoder module receives the streamed data from the microcontroller and decodes it, resulting in a small signal audio output.

The Microcontroller and Software

The microcontroller chosen for this application is the Atmel Atmega 328P. This processor is simple to use, relatively inexpensive and more than meets the requirements for this portion of myDoorbell. The schematic below illustrates the circuit used for this application. The labelled input and output signals are grouped into several main functions:

  • Input from the doorbell button detection circuit described in the previous post.
    • FBELL_BUTTON and RBELL_BUTTON
  • Output signals to the second microcontroller.
    • FBELL_EVENT and RBELL_EVENT
  • Serial Peripheral Interface (SPI) signals for communicating to the MP3 decoder and programming this microcontroller
    • BELL_SCK, BELL_MISO and BELL_MOSI
  • Memory control and data signals
    • MEM_SEL, MEM_GRANT, MEM_REQ, MEM_RESET, MEM_RDY and MEM_SB
  • MP3 decoder control and data signals
    • VS_XCS, VS_XDCS, VS_DREQ, VS_RESET and MUTE
  • An external clock running at approximately 8 MHz.
    • EXT_CLK 

The LED and the associated current limiting resistor are available as a programmable output that is easily visible. This can be used to indicate state or for initial debugging. The push button enables the device to be manually reset. The capacitors are 0.1 µF devices that act as bypass capacitors, see Basics to Remember below. The six pin header, in the upper right corner of the schematic, is for programming the microcontroller.

The software that drives the signals identified above can be found in the myDoorbell repository on github.com. While there is considerable amount of detail, that is best addressed by viewing the source code, a brief overview is useful.

The “main” routine initializes the memory module and the MP3 Decoder module by communicating with them over the identified control and data lines. This communication takes place using the SPI protocol. This routine then loops checking the state of the front and rear doorbell buttons and requesting or relinquishing control of the memory device to / from the second microcontroller.

If the front or rear doorbell buttons are pressed, the microcontroller accesses its built in EEPROM to determine the volume at which the ringtone should be played. It then plays the appropriate ringtone for the front or rear door.

The ringtone is played by determining its length and repeatedly reading portions of it from the memory and streaming it to the MP3 decoder. When the data is consumed the device goes back to checking the state of the doorbell buttons.

The MP3 Decoder

The MP3 decoder is based on the VS1011 MP3 Audio Decoder. The following schematic is recreated from it datasheet.

The input and output signals, on the left of the schematic, are connected to the identically named signals from the microcontroller schematic already presented. The outputs on the right are the two audio output signals and the common between them. Lots of bypass capacitors and a few other components, as illustrated in the datasheet, are included.

The VS1011 is capable of consuming MP3 and WAV formatted audio files and generating low level audio output. The audio output is sufficient to drive a pair of headphones, but in our application will drive an audio amplifier and associated speaker.

A Failed First Attempt 

The first design had a serious flaw that only manifested itself when in use. This misstep led to the design described in this post. However, since this design was put into use, two others have been proposed and will be described later.

The first design had a single microcontroller that interacted with the WiFi module, the doorbell button circuitry, the memory and the MP3 decoder. There was a major issue with this approach. When the doorbell button was pressed, should the MP3 file be played before an event is sent to the user, such as a text message, or should the event be raised and then the ringtone played?

If the event is sent first it will delay the playing of the ringtone by at least hundreds of milliseconds, but often much more. This doesn’t sound like much, but when you push a doorbell button you expect to hear the bell. In practice it was very odd to push the button and not get immediate feedback.

If you play the ringtone first and then send an event it works great, unless someone picks a 30 second ringtone. In this case, by the time you receive the text message and arrive at the door, the person is long gone. I have personally found that this behavior is often a feature, but others disagree.

This issue led to the design described here where we have two processors that communicate via shared signals and a shared memory. This approach is obviously more complex and more expensive.

Physical Implementation

The schematic for the microcontroller shown above resulted in the following PCB layout.

This design required a few traces to be routed on the bottom layer. As with previous PCBs illustrated in this series of posts, this one also utilizes a 4-layer design with internal layers carrying ground and 3.3 V. The reset button is on the far left, the microcontroller in the center and its programming header on the right.

The schematic for the MP3 decoder, illustrated above, results in the following 4-layer PCB. The crystal that controls the clock for this unit is located on the left of the layout. Most of the input and output lines are located at the bottom of the board. The low level

audio output interface is located on the right. As in previous layouts, bypass capacitors and other components are placed as close to the device as possible. 

Two Alternative Designs

The first alternative design is based on the initial single CPU misstep. In the first design the problem was dealing with which came first, the raising of external events or the playing of the ringtone. A better approach may be to send the MP3 decoder the first 32 bytes of MP3 data and then start raising the external event. The MP3 decoder has a data request (DREQ) line that gets asserted whenever the decoder is capable of receiving another 32 bytes of MP3 data. If this were tied to the microcontrollers interrupt line it could be used to direct the microcontroller to send more audio data and then return to raising the event. The only question is whether the 8 MHz microcontroller is fast enough to switch between the two tasks and accomplish what needs to be done without introducing gaps in the ringtone.

Back of the envelope analysis to figure out if this would work goes something like this. Good quality stereo MP3 files are about 1 Mbyte per minute in size. Since we only need one channel we can reduce the file size to 500 Kbytes per minute. We can clock SPI data into the MP3 module at 4 MHz or 4 Mbits per second. With 8 bits per byte this translates to 500 Kbytes per second. With switching time between tasks and other overhead this won’t work, but it is likely that we can get by with far less quality than we expect from our digital stereo music. Reducing the quality will likely make this technique possible.

The second alternative design takes a radically different approach. For this approach we trade out the WiFi module, both microcontrollers, memory, and the MP3 decoder and replace it with a fast embedded ARM microcontroller that runs the Linux operating system. An example of this is the Onion, shown below.

The microcontroller runs at 400 MHz instead of 8 MHz, giving us plenty of horsepower to accomplish our task. The Linux operating system provides the task switching necessary to switch rapidly between raising external events and playing ringtones. Audio playback software can be used instead of the dedicated MP3 decoder hardware currently being used. Finally, this sort of board costs less than $20, which is less than the WiFi module used in the current approach. This approach is currently being pursued.

In my own defense let me state that these sorts of modules were not available when the first design was completed. In addition, education leads to doing things in a better way; I’ve learned a lot!

Outcomes and Artifacts

There are likely many ways to build this portion of myDoorbell and a few of them have been described. The Eagle schematic and PCB files are available from the myDoorbell repository at github.com.

Basics to Remember

Bypass capacitors are used in many digital systems. Each clock cycle a digital system has the potential to change from one state to another. When this occurs the device draws significantly more current for a very short period of time. Recall that every wire acts like an inductor and that inductors resist changes in current. If the PCB trace from the device to the power source is long or narrow it will restrict current sufficiently to starve the device from the energy that it needs during these transition periods. Bypass capacitors hold sufficient charge to supply the device with the needed energy during these short periods of time. They are recharged while the device is relatively stable. These capacitors should be located as close to the device as possible and located between the power source and the device. This close proximity will reduce the inductance of the connecting trace.

myDoorbell: The Button

Introduction

In my previous post, I described myDoorbell’s power supply module. In this post I will describe the circuitry that translates a standard doorbell button press into something that can be easily consumed by a microcontroller.

Standard Operation

The standard doorbell circuit is shown in the following figure. Recall that when the doorbell button is not pressed (open), current flows from the transformer, through the lamp, and through the solenoid, completing the circuit. With the doorbell button open, my test setup has approximately 50 mA of current. When the button is pressed (closed) a larger current, approximately 1 A, flows through the circuit.

myDoorbell is complicated by the system requirement to use the existing doorbell button and retain the functionality of its lamp.

myDoorbell Button

The figure above can be represented by a schematic where the lamp is replaced by a 320 Ω resistor. We model the lamp with this value resistor because with 16 Vrms across it,  the current flow is ~50 mA.

myDoorbell does not need the solenoid in the above circuit. However, if we simply replace it with a wire we will create a short circuit when the button is closed. In this case, approximately 0 Ω will exist, resulting in nearly infinite current, a bad idea. So replacing the solenoid with a resistor, to limit current, is necessary, but what value still needs to be determined. A large value resistor will result in little current, which is attractive, but too little current and the lamp won’t be bright enough to be useful.

As previously discussed, resistors have resistance values measured in Ω’s, but they are also rated for the amount of power they can dissipate. The power dissipated by a resistor is equal to the product of the voltage across it and the current through it. Through manipulation of Ohms Law it can be shown that the power dissipated by a resistor is also equal to the product of its value and the square of the current through it. With this in mind, if we choose a low value resistor its power rating will need to  be increased.

For example, if we choose a 10 Ω resistor, for the top resistor in the above figure, then approximately 48 mA will flow through the lamp when the button is open. The 10 Ω resistor will have to dissipate 0.02 W, well below the 0.1 W rating typical for small surface mount resistors. However, when the button is closed 1.6 A will flow through the circuit and the resistor must dissipate 25.6 W, it’s going to catch fire!

Let’s choose a higher value resistor, say 10 kΩ. Now when the button is open approximately 1.6 mA of current flows through the circuit resulting in the 10 kΩ resistor dissipating just 0.02 W, which is great, but the lamp will be very dark, bad!

So let’s compromise. We’ll choose a 220 Ω resistor that results in a current flow of 29 mA and a lamp that is visible when the button is open. In this mode the 220 Ω resistor will dissipate 0.185 W. This will require a larger resistor, but manageable. When the button is closed the 220 Ω resistor results in a current of nearly 73 mA and a power dissipation of 1.17 W. This will require quite a large surface mount resistor, but is necessary to meet our design requirements.

Detecting the Press

When the button is open, the voltage across it is equal to 320 Ω * Ilamp. Where Ilamp is equal to 16 Vrms / 540 Ω, or 29 mA. So the voltage across the button when it is open is 9.48 Vrms. When the button is closed, the voltage across it is 0 V. What we need is a circuit that detects this difference and generates a DC voltage appropriate for an input to a 3.3 V microcontroller. The figure below depicts just such a circuit. 

To the left we see the portion of the circuit described earlier, consisting of the transformer, push button, the 320 Ω resistor representing the lamp, and the 220 Ω resistor added to restrict the current flow when the button is closed. The circuitry to the right needs explaining.

Let’s start with the component outlined in the rectangle, an optocoupler. This device consists of a LED that emits infrared light onto a light sensitive driver. When the input current is sufficiently high, infrared light is emitted by the LED, and the output of the driver is pulled to ground. When the input current is sufficiently low, the output is left in an “open collector” state where no current flows. With no current through the output resistor, there is no voltage drop across it. In this state, the output level is equal to the voltage attached to the other side of the output resistor, in our case 3.3 V. A representative device is the H11L1M. Its datasheet can be found here.

In our application, when the doorbell button is closed there is 0 V across the LED of the optocoupler and the output will be 3.3 V. When the button is open the 320 Ω resistor, representing the lamp, will have 9.48 Vrms across it. The peak voltages are -13.4 V and 13.4 V. According to the optocoupler’s datasheet, its LED cannot tolerate reverse voltages exceeding 6 V, so the external diode is added to protect it. The external diode conducts when the input voltage is negative, resulting in roughly 1 V across the LED. 

Again referring to the datasheet, we need 1.6 mA of forward current on the LED to get the device to activate, driving the output low. As the input voltage rises from 0 V to 9 V we’ll allow the output to remain high, but when it arrives at 9 V we’ll switch the output to low. This will result in a 60 Hz sequence of 3.3 V pulses while the button is open. To accomplish this we need the optocoupler input current to be less than 1.6 mA with input voltages less than 9 V, and greater than 1.6 mA when the input voltage are higher.

Imagine we don’t know the value of the 5600 Ω resistor in the above circuit. To meet the requirements described in the previous paragraph, we need to size this resistor appropriately. If there is 9 V on the left of the resistor and 1 V on the right side, due to the forward voltage drop VF across the LED, there is 8 V across the resistor. This is explained by Kirchhoff’s Voltage Law that states that the sum of voltages around a loop must be zero. To get 1.6 mA at 8 V we need a 5 kΩ resistor, we’ll use the more common 5600 Ω device. At our peak voltage of 13.4 V, we’ll have 2.2 mA of current through the optocoupler’s LED, well below the 30 mA maximum value the device can handle. 

If the optocoupler’s output signal is tied to the input line of a microcontroller, a simple software missing pulse detector can be used to detect when the doorbell button is closed, see the sample below. In this example we check the value of the optocoupler’s output once each ms for one 60 Hz cycle, or one 17 ms period. If it is ever zero the button is open.

Physical Implementation

The complete schematic, with component values, power ratings, and tolerances, is shown below. Component tolerance describes the range of possible actual values of a component as a percentage of their ideal value. If our 5600 Ω resistor has a 10% tolerance, then a real device may have a resistance value as low as 5040 and as high as 6160, with the actual value being somewhere between these extremes. It is important to take these minimum and maximum values into consideration during a design.

Our design is sensitive to the value of R3, the 5600 Ω resistor. For this reason we chose a device with a 1% tolerance. This costs a bit more, but in this case it was important. Also note that we chose a 3W resistor for R1 to ensure we don’t have a catastrophic failure, a fire!

The printed circuit board layout for the above design is shown below. The pads on the far left are for TB1 which is a connector where the two wires from the doorbell button are connected. Notice R2 in the lower right hand corner of the PCB. It is the 680 Ω output resistor and is a normal sized surface mount resistor. Compare this to the size of R1 which is the 220 Ω 3 W resistor, quite large in comparison.

This circuit results in a simple layout that doesn’t require a 4-layer PCB. However, to simplify connections to other myDoorbell modules, a 4-layer technology was used. No routing on the bottom of the board was required. The inner layers are used for ground and 3.3 V and are brought to the top layer using the three visible vias.

When integrated into the final system, small changes were made to optimize the overall layout. In the above photo of the myDoorbell button detection module, you’ll notice that the two smaller resistors and the diode have been move to slightly different locations. The final system contains two of these modules, one for the front door and the other for the rear door.

Outcomes and Artifacts

This post describes a simple circuit and associated software that meets the requirements of keeping the doorbell lamp functional and detecting when the button is pressed. There are likely many alternative techniques for meeting these requirements, but one worth considering is to use an AC line monitor chip such as the MID400 which is intended to raise a signal if the power line voltage is interrupted. In our scenario it would raise a signal if the AC voltage across the doorbell button was absent, or in other words, the button is pressed. This unit costs slightly more than the circuit described above, but would eliminate the need for the external diode and the missing pulse detector software. The output could be tied to a microcontroller interrupt line making the system more responsive, reliable, and perhaps simpler.

The Eagle schematic and PCB files are available from the myDoorbell repository at github.com.

Basics to Remember

The power dissipated by a resistor is equal to the product of the voltage across it and the current through it, P = VI. However, we often don’t know the voltage across a resistor, but rather know the resistance value and the current through it. In this case a simple manipulation of Ohm’s Law yields a solution, P = I2R.

The sum of voltages around a circuit loop must be zero. This fact is known as Kirchhoff’s Voltage Law. It is useful for circuit analysis and as used in this post to determine the proper circuit element values to meet requirements.

Indie Educational Technology

Introduction

Last week I spent some time at Davidson College discussing Personal APIs and Indie Educational Technology with faculty, students, and staff from several other institutions of higher education and commercial entities, Known and Reclaim Hosting, that facilitate this work. This was a fantastic gathering of bright people and I can’t wait to be with them again. Thanks to Kristen Eshleman for getting us together, to Ben Werdmuller and Erin Richey for their instruction, to Audrey Watters for her insightful description of indie, and to Tim Owens, Jim Groom, Kin LanePhil Windley and Troy Martin for always making me think better. After returning to my day job I found myself asking the question, what is “indie educational technology”?

According to CNN “If it’s cool, creative and different, it’s indie” and the Urban Dictionary defines indie as, “an obscure form of rock [music] which you only learn about from someone slightly more hip than yourself.” I had to travel to North Carolina to have people, more hip than myself, educate me! I thank them and hope they continue to help me progress.

While I’m pretty sure the term “indie” in this context meant “independent of vendors and personal”, for my purposes I’m going to define indie educational technology as information technology that is cool, creative and different used to enhance the educational process.

Indie technology benefits both students and educational institutions. Students have a greater sense of ownership and motivation when they (a) control their personal information and (b) are able to interact with institutions with both institutionally provided applications and alternative systems (McCombs, 1997). Institutions benefit by having alternative application-hosting options and are not unnecessarily burdened by housing personal student information with its associated liability.

For example, let’s consider a traditional, non-indie, university registration system. One or more centralized systems contain university information about courses, classrooms, and instructors. In addition to this information, these systems contain personal student information that students are required to submit to participate in the registration process. To register for classes, students present user credentials to the university provided registration system, register for classes, and end their interaction. In this model the university retains personal student information, insists that students use the university provided system, and refuses to make alternative systems possible or feasible.

Consider an alternative, the indie approach. Like the non-indie system described above, one or more centralized systems contain university information about courses, classrooms, and instructors. However, personal information is not retained in this system. A student’s personal information is housed in a student-controlled system. Students authorize the university’s system to access necessary information. In this case students register for classes by presenting credentials to a registration application of their choice. This registration system requests and receives authorization from the university and the student to acquire university and personal student information, respectively. They register for classes and end their interaction. In this model the registration application disposes of personal and university information it was exposed to, while the university system retains the information necessary to indicate successful registration. In this scenario students may choose to use the university provided registration tool or alternative systems. In addition, the university does not house personal information or bear the associated liability. That’s cool, that’s indie!

So what makes this technology cooler, more creative, and different from what currently exists? Let me suggest that it is because indie technology will have several characteristics:

  • Personal API Enabled
  • Substitutable
  • Open Source
  • Modular
  • API-Based
  • Event-Driven

Let me elaborate on each.

Personal API Enabled

A personal API (PAPI) is an interface to personal information and resources. The resource owner protects these resources through explicit authorization. There are at least three key benefits of developing and using a PAPI:

  1. A PAPI changes the expectations of users. They develop a sense of ownership of their information and resources and begin to expect institutions to respect their rights and privacy. The use of a PAPI at an institution of higher education yields a perfect opportunity to educate students about these issues and help them understand what they should expect from other vendors and providers. They benefit from the ability to disassociate from institutions by simply revoking authorization to their data.
  2. A PAPI eliminates the need for a single university or vendor-provided application that all users must interact with. Users interact with their PAPI using applications of their choice. Institutional systems request permission to access personal information through the PAPI to perform needed functions. Institutions may provide applications for users, but the PAPI facilitates the creation of alternatives.
  3. A PAPI frees institutional technology modules from having to unnecessarily hold personal information. Institutional systems request needed personal information through the PAPI to perform their functions. Institutions should find this attractive because they will house less personal information, reducing their liability.

Note that giving people a personal API and letting them control their data, doesn’t mean that they get to control the university’s data. A PAPI lets people control the data that is theirs. For example, their phone number is their data. Their grades, on the other hand belong to the University. In addition, if students exercise their right to not authorize university access to needed personal information, the university is not obligated to fulfill the desired student request. University policy and process must still be followed.

The institutional complement to the PAPI is what we call the University API (UAPI). Through the UAPI an institution protects its resources through explicit authorization. In our example indie class registration system, the UAPI would make course, classroom, and instructor information available to the student chosen registration application. In addition, the UAPI would record the necessary registration outcomes.

Substitutability

Substitutability is the ability to use alternative systems or services to accomplish specific functions and move from one platform to another with ease and at little expense. This is applicable to both users and institutions.

Substitutability benefits users by allowing them to move their systems and services to alternative providers. They are also free to choose alternative applications to perform functions of interest. Institutions should facilitate both by pursuing strategies that allow authorized access to institutional information and consume necessary personal information through a PAPI respecting the user’s expectations of privacy.

Institutions benefit as well. Their systems can be operated on multiple platforms and through the use of technologies such as the UAPI and the PAPI, alternative systems can be used to accomplish institutional functions. If institutional systems only work properly when hosted at a single provider or moving them is onerous, institutions leave themselves vulnerable and open to the policies and practices of that provider. The inability to easily substitute one provider for another brings us back to our current state of affairs.

Open Source

Created systems and services should be freely available to others. First, this is what the cool kids do – indie. Second, by making them and API definitions freely available, others are more likely to adopt the technologies. Wide adoption results in many smart, hip people working on the same problems, resulting in better solutions. Licensing them appropriately protects our ability to use the things we develop.

Modularity

Modularity facilitates and drives an increased pace of innovation. Each module should deliver a small set of functions within a single bounded context as defined in the domain-driven design process. While these modules can be created using various techniques, at Brigham Young University (BYU) we will be defining them as microservices. These microservices will result in stand-alone modules that are easily understood by developers and will encourage extremely loose-coupling, facilitating a building block mentality to building systems. This approach will drive innovation in the core processes of BYU. 

API-Based

Each module will have an API that enables communication to and from the module. The API simplifies the use of the module and abstracts away the internal implementation. This abstraction permits changing the underlying implementation while protecting systems that rely on the module’s API. 

Event-Driven

While not strictly necessary, event-driven architectures are more efficient and absolutely cooler than polling-based systems. I think this alone makes event-driven, modular design a part of indie technology!

In a polling-based system you only become aware of changes when you ask if changes have been made. For example, in a registration system you determine how many students have registered for a particular class by asking (polling) the system. In an event-driven architecture, each time a student registers for a class an event reflecting this activity is posted to interested listeners. This results in more efficient communication and more timely responses to change. What could be more indie?

Now What

At Brigham Young University we intend on building many, if not all, of our core academic systems and services using modules with the above characteristics. The result will be a collection of modules that perform core functions of the institution, but are likely usable by others.

I hope that we can find ways of including others outside of the BYU community in the creation of our functional modules, systems, and services. Including others will make our work better, but more importantly will result in definitions and implementations that are more generic, enabling others to use them more easily. Each module, system, and service will have the characteristics outlined above making their use elsewhere practical and possible.

Finally, I hope we can all find a way to meet regularly to showcase our attempts, failures, and successes. We, at BYU, are open to conferences, workshops, or other venues where we can all continue this discussion.

myDoorbell: High Level Design

Introduction

In my previous post I described the resources available in the wiring box where myDoorbell will be installed. As illustrated in the following figure, the available resources consist of a 16 Volt AC source and a connection to a doorbell button with an integrated lamp.

When the button is pressed, approximately 1 Ampere of current flows through the circuit. This current at 16 Volts AC yields 16 Watts of available power (16 Volts * 1 Ampere). This is sufficient for anything I am envisioning. For a more thorough, yet simple, explanation of Ohm’s Law click here.

Features and Hindsight

As pointed out in my first post on this subject, myDoorbell will have several unique characteristics or features:

  • Play ringtones uploaded by the user to celebrate seasons, holidays, birthdays, etc.
  • Record the time and date of each ring
  • Be configurable
  • Send a text message indicating a ring, if configured to do so
  • Be silent if configured not to make noise when babies are sleeping, pets shouldn’t be disturbed, or the owner just doesn’t want to know visitors are present
  • Access other Web resources such as APIs, webhooks, etc. 

As the saying goes, hindsight is 20/20. If I were designing myDoorbell all over again, I actually am, the next thing I would do is design the API that would describe the HTTP resources, methods, and error codes used to implement the above features and behaviors. However, when I began this adventure I didn’t know much about APIs, HTTP, and related technologies. Recall that this lack of knowledge was precisely why I started this adventure.

I will proceed in roughly the order I actually took, but want to encourage any readers creating hardware or software systems that they should design the API first. Since software is eating the world it is clear that any real system will contain software, and I believe that all such systems should have an API. Creating the API first will result in a better product and will reduce the risk of cresting hardware and software components that in the end are unnecessary.

Necessary Components 

Several hardware and software components are necessary to implement the characteristics and features listed above:

  • myDoorbell needs Internet connectivity to facilitate the uploading of ringtones, to enable the calling of other APIs or to raise events, and allow the acquisition of configuration information and updates. Since there is no wired network connection in the doorbell wiring box, this connectivity will be provided by a WiFi module. 
  • A microcontroller is needed to interact with the WiFi module, provide a compute platform for the web server implementing the myDoorbell API, interface with a nonvolatile memory, and enable the detection of the state of the doorbell button. 
  • A software module is needed that time slices between detecting the state of the doorbell button and dealing with incoming network traffic. It must also copy incoming ringtone data to the nonvolatile memory. Finally, this module must implement outgoing requests and the raising of events.
  • A simple hardware module is needed to provide doorbell button state information to the processor.
  • A nonvolatile memory is needed to store ringtones and configuration data. This memory needs to be writable by one microcontroller and readable by another, two ported.
  • A second microcontroller is needed to copy ringtone data from the nonvolatile memory to the audio system. 
  • An audio subsystem is necessary that converts mp3 encoded music to audio and amplifies the resulting audio to a sufficient amplitude to drive a speaker to the required volume.
  • A power supply is needed to convert the 16 Volt AC to a 3.3 – 5 Volt DC source for the digital electronics and a higher DC source for the audio amplifier. 

The following figure is a block diagram illustrating the hardware components listed above. Note that there are two modules for detecting the doorbell button state because most commercially available doorbells have the capability of ringing for both a front and rear door.

There are of course many ways to design any system, and I am not arguing that this is the best or even a reasonable design. In addition, as technology changes so do the choices and possible optimizations. My intent was to learn, and I learned a lot. In my next post I will describe the power supply module. This seems simple, but hang on and keep your arms and hands in the ride at all times!

myCoreDump

Introduction

I hope you enjoy this core dump! The thoughts are so interrelated and connected it is difficult to optimize the presentation so you may need to apply your own defragmentation to get it. In addition, the order is not intended to indicate priority, it is all freaking important!

University API (UAPI)

When acquiring or developing an application it must have an API, and preferably a RESTful one. If the function of the application is core to university business then it should be exposed through the UAPI. If it is not a core function of most, if not all, educational institutions, we should expose the API through our API management tools, but it shouldn’t be part of the UAPI.

Personal API (PAPI)

When we build a system that will store personal / individual information we should consider how we might leave the information in the hands or possession of the individual and access it for our use through their personal API. Since no one yet has a personal API, for the time being we must provide that as well. This will require you to stretch your imagination and creativity, but that’s good for you.

Domain-Driven Design

Everyone should read at least the first two chapters of the book Implementing Domain-Driven Design by Vaughn Vernon! The super short summary – bring domain experts and developers together to create a ubiquitous language that is embedded in the code itself. In addition, define or determine bounded contexts wherein this language is valid. Without this you won’t understand how we’re going to build solutions and you won’t have a clue what is in and what is not in a microservice. Read it!

Microservices

Microservices are an architectural style that will be used at BYU to create larger systems. Systems built using microservices are loosely coupled, I would even go as far as saying they are highly decoupled, they implement a single business capability, they have well defined interfaces, and communicate using only these interfaces. The size of a microservice is governed by the size of the associated bounded context, go and read the DDD book! At BYU an important part of a microservices’ interface is its ability to raise events. Go figure out why.

Event-Driven Architecture (EDA)

Systems that poll are inefficient! Build systems that raise events so other systems don’t have to waste time and resources. You can keep asking me if you have to do this, but you can be assured that when I change my mind I’ll let you know. If you didn’t find the humor in the last sentence then go read the links again.

Application Acquisition

When we purchase applications we should give preference, strong preference, to those that run in the cloud. In fact, before we choose an application that is not available as a service choose someone in your group you don’t love and care about to come get my approval.

When we build services or applications they will run at Amazon and use the most abstract service offerings that make sense. In other words, we should not instantiate EC2 servers and S3 storage and then build queues, notification services, etc., but instead should use services such as SQS, SNS, Lambda Functions, etc.

DevOps

DevOps is a culture and practice that we hope will result in the rapid development, testing, and deployment of software. We are measuring the number of deployments / week, failures / week, and time to recovery. We are promoting small changes, thorough automated testing, and deployment to production often. Your team (the DDD team) is in charge and responsible for the functionality, performance, and reliability of “your” product. 

If those in the hardware world think you’re off the hook, think again. Software is eating the world, software is eating your world. The days of interacting with network switches, routers, firewalls, etc. are over. Learn to program, learn to configure hardware devices using programs, learn to use DevOps to configure, test, and deploy hardware platforms as rapidly as “other” developers – that’s right, you just became developers!

Where to Compute

In the past we built data centers and populated them with servers, storage systems, and network components. As CPU performance increased computers became more able to run multiple applications, but stability due to unintentional application interaction made this approach intolerable.

We found ourselves with many underutilized servers running single applications to maintain reliability. Along came server virtualization enabling us to instantiate multiple virtual servers on each physical server. Over the past several years the number of physical servers has diminished considerably.

Well, it is time for another paradigm shift. We are now embarking on a journey that will result in our compute and storage being somewhere else. We will take advantage of Amazon to deliver what our applications and services need to run. Acquired applications will also run in the “cloud”. in either case they will not be housed here. Resources used previously to purchase servers and storage, and maintain them will be redirected to this new endeavor.

Networks

Unlike server and storage, I believe we will have a wired and wireless network on campus for the foreseeable future. However, the way we deploy, configure and maintain these networks will change drastically. Remember, software is eating the world and networking is not an exception to the rule. Network components will be physically installed in some generic way and then configured remotely via software.

In a DevOps fashion, when a problem occurs you figure out what went wrong in the configuration script, you repair the script, you test the script, and you redeploy. Remember, we’ll be watching how often you deploy, how many failures occur, and how long it takes to recover.

The days of hugging these devices are over. If you want one to hug, you can have one of the old ones and keep it in your office – disconnected from the network of course.

Domain of Ones Own (DoOO)

As we embark on this new path it is a great time for you to consider contributing to the content of the Internet. Let your light so shine by getting a domain of your own and sharing your goodness and skills with others. get one at domains.byu.edu. Here you can blog your greatest thoughts, post content that you syndicate to Facebook, Twitter or other services. Go learn, learning is fun!

We are offering this service to all students because we believe they should understand more about how the Internet works. We believe they have much to offer the world and they need to know they can share it with little help from service providers. What they build is transportable to other hosting services and is theirs! In the future a DoOO will enable an individual to have a portfolio and expose this and much more through their personal API (PAPI).

Final Thoughts – For Now!

We have a great team! Let’s pursue all of this FUN with the greatest enthusiasm and Heaven will shine down on us. Let us share our best thinking with others: share code on github, answer questions on stackoverflow, blog about your experiences, publish papers, present at conferences, participate on panels. In short, learn, teach one another, and teach the world!

myDoorbell: A Learning Adventure

Introduction

After being a university chief information officer (CIO) for more than a decade, I decided to refresh my technical skills acquired through formal education and practice as an electrical engineer. I learn best by doing, so I picked a project I was interested in pursuing with the end goal being the learning, and not the finished product. I intend to share several posts that I hope illustrate the things learned and hope they are of value to the reader.

My Project

I have interest in the Internet of Things (IoT) movement and wanted to make strides towards making this practical, simple, and secure. I believe connected devices should be simple and consume little power. This likely requires devices that wake periodically, connect to some sort of network, and then go back to a low power state. After some experimentation it was clear, at the time, that WiFi was a real power hog and wasn’t a likely candidate. However, this realization led me to believe that another router, hub, or coordinator device would be necessary. I recall the effort required to convince homeowners to acquire WiFi routers and looked for an approach that would make this palatable.

I decided the answer was to create a product that homeowners would want to purchase because it excited them, and by the way it contained a network router / coordinator. Once acquired on its own merits, the product would facilitate the inexpensive and simple acquisition of other devices that connect to it. Products worth considering would be interesting to households and would connect to household power:

  • Lamps
  • Televisions or other audio / video (AV) equipment
  • Thermostats
  • Doorbells

Lamps seem simple and boring. However, after implementing my first choice, I know I should have chosen a lamp because it would have been boring, simple, and done! I decided embedding anything in televisions or other AV equipment would require skills and resources I didn’t have. Nest took the thermostat direction and while I disagree on the approach of putting so much technology in a tightly coupled system, I didn’t want everyone to judge my work against a commercially available product. I chose to implement a doorbell because they are ubiquitous, simple and meet my requirements:

  • They are in nearly every U.S. household.
  • Power is available where the indoor ringer is found.
  • They do one thing and no one cares if they do anything else.
  • They are in a good physical location for a network router.
  • They are out of the way, aren’t moved, never unplugged, or inadvertently reconfigured.

I chose to create a doorbell that would function as a replacement doorbell, would act as an IoT network router, and connect this network to the Internet by also connecting to an existing WiFi network. A quick trip to Home Depot revealed that an inexpensive doorbell cost about $13. Even with no experience in product development, I knew I wasn’t going to be able to build a doorbell that also acted as an IoT to WiFi gateway for $13. To be compelling enough to get households to acquire my doorbell it would have to be feature rich:

  • This doorbell would play ringtones uploaded by the user to celebrate seasons, holidays, birthdays, etc.
  • Each time someone rings this doorbell the time and date should be logged.
  • The owner can configure the bell to text them when someone rings.
  • The bell should be easily configured not to make noise when babies are sleeping, pets shouldn’t be disturbed, or the owner just doesn’t want to know you’re there.
  • When the bell is rung it should be configurable to access other Web resources such as APIs, webhooks, etc.
  • The system should be controlled and configured using a mobile app.
  • The doorbell must be a simple replacement of the original doorbell ringer.

While these features increase the likelihood of making it compelling enough to overcome the necessary price point, it certainly eliminates any chance of it being simple.

Summary

In this post I declared my intent to refresh my technical skills through the development of an IoT product, an amazing doorbell, myDoorbell. In the next few posts I will describe how a typical doorbell works, illustrate the general system layout for this new doorbell, describe how to create it so it fits into existing doorbell systems, and discuss many details of the techniques and technologies that make this possible. It will be a fun journey with many twists and turns, but that’s how learning happens!

Domains, Personal APIs, and Portfolios

Introduction

In addition to the traditional educational experience students at Brigham Young University receive, we want them to acquire skills, techniques, and tools that facilitate their current and future learning. We believe students should learn how to control and own their digital identity, content, and personal data. With this goal in mind we have initiated a pilot program using a concept known as Domain of Ones Own. We hope to accomplish several goals using this concept and associated training:

  1. Teach students, faculty, and staff why they should care about owning, controlling, and appropriately sharing their online identity, the content that defines them, and their personal information.
  2. Help individuals understand how to choose a domain name that accurately and professionally represents them to others.
  3. Encourage members of our community to not simply consume, but contribute to the body of knowledge through the use of blogs and social media.
  4. Support individuals in publishing a Personal API (i.e. api.example.com) that allows the owner to authorize others to interact with their personal information and revoke access privileges as desired.
  5. Support students and faculty in creating a portfolio (i.e. api.example.com/portfolio) as part of their Personal API that is owned and maintained by the individual owner, and yet enables the owner to authorize others to consume, contribute to, and evaluate their collection.

Domain of Ones Own

Many members of our community share their pictures, memories, thoughts, insights, and writings on social media sites that are controlled by others. The privacy policies of these sites change over time, access privileges may change, copyright ownership is a concern, and the look and feel desired by the content owner may change without their knowledge, input, or control. Contributors have no control over the amount or type of advertising placed around or even over their content. In many cases they may not be able to easily move their content to other providers, remove content they no longer wish to share, or even pass ownership onto others as desired. We want members of the BYU community to understand that there is a better way.

Consequently, we have chosen to use and teach a concept known as Domain of Ones Own. We first herd about Domain of Ones Own from Jim Groom when he was at the University of Mary Washington. After a visit we were hooked on the idea of freeing our community and using the tool to rethink content ownership, Personal APIs, portfolios, and Learning Management Systems.

Our implementation of a Domain of Ones Own consists of a simple hosted server configured using cPanel and pointed to by the end-user’s chosen domain. We are using the service and tools provided by Reclaim Hosting who provides the tools, hosting, and the process for acquiring domains. With the default, initial configuration domain owners have a blog driven by the Known blogging tool. While this is a great introduction that allows domain owners to contribute immediately, the system is open and can grow as the domain owner’s sophistication increases. The system allows users to set up subdomains, email servers, database servers, and install and run many LAMP stack based applications. The tools and services have been chosen carefully to allow users to move their domain and associated content to other providers easily. Tools were chosen to be immediately useful, provide future flexibility, and help users learn introductory system administration skills that are critical to understanding the world they are in and will inherit.

Domains

We believe every individual should own and control their domain. Choosing an appropriate domain is important. In many cases the domain will be used in a professional capacity for years, perhaps for life. We are creating instructional material, including short video segments, which will give advice on how to choose well. We intend to create these materials in a way that minimizes branding and IP protection so others can easily use them for similar purposes at their institutions.

Personal API and Portfolios

Imagine a world where other sites on the web don’t hold your personal data, but instead request access to the data they need through your Personal API. Perhaps you grant them access to only the portions they actually need and restrict them from others. They use the resources they’ve been authorized to access, perform the business functions you desire, return results, and their access is revoked.

For example, imagine you work for weLovePrivacy.com and it’s payday. The payroll system springs to life and determines how much you should be paid this month. However, it needs to know how much should be withheld for taxes, how much pretax contributions to make, where these should be made, where you want your money deposited, etc. In a traditional system all of this information is centrally held. This centrally held information compels the institution to create systems to enable you to manipulate it, and makes the company liable for any loss of this data. On the other hand, you are depending on the institution safeguarding your personal information and not using it for nefarious purposes, a dangerous assumption.

However, there is a better way. Imagine the payroll system interacts with your Personal API to obtain your social security number, the number of exemptions you are declaring, the name of your 401k vendor, 401k account number, your checking account provider and account number, etc. The institutional system does the computation and disbursements, and your Personal API revokes access to these resources until the next time they are needed. While the institution could store the collected information it may not be in their best interest to do so and could even be released to them with the understanding it is to be used for the sole purpose disclosed to the user.

While it may be a while before ERP administrators are comfortable getting employee data from their personal API, there are plenty of other scenarios where a personal API is useful. Portfolios is an example of such a scenario. An instructor at an institution requests authorization to place assignments into your Personal Portfolio, their request is granted, and the assignments are deposited. You perform learning activities that generate solutions to the assignment, and deposit these in your portfolio. You have authorized the instructor to see them and place their critiques back into your portfolio. Since this is your portfolio it moves with you from one part of your life to another, from one institution to another, etc. It is yours to use and share as you choose.

Summary

It is time for learners to take control of their content, artifacts of education, and personal information. Our desire and intent is to teach these principles to our community and give them the necessary tools. We hope to do so in a way that others can easily use and benefit from.

Freedom via Abstraction

At Brigham Young University (BYU) we have been developing a University API to expose the functionality of a reasonably generic educational institution, while consuming a very specific set of underlying technologies. Our generic institution has instructors, students, courses, classes, and locations. These resources and available HTTP methods are being combined to expose acceptable business processes such as registration, adding and dropping classes, etc. We will continue to add resources and appropriate business processes as necessary to meet our institutional needs.

Our intent is to develop future applications by consuming the University API and will encourage others to do the same. We will no longer consume the user interfaces or APIs of underlying systems. This layer of abstraction will enable us to replace the underlying technologies with new technologies that provide similar functionality. Regardless of the tools or technologies used, those consuming the University API will be unaware of the underlying change. This will give the IT organization the freedom to make changes to reduce cost, modularize monolithic applications, move to microservices, etc. without impacting application developers or end users. This will bring them freedom via abstraction.

I’m writing about this today because in my mind this is an important general architectural pattern that should be followed more often. David Wheeler­­­, a British computer scientist, is credited with saying, “All problems in computer science can be solved by another level of indirection, except of course for the problem of too many indirections.” While most often quoted by programmers in discussions about pointers and similar constructs, I think abstraction layers, like the one discussed above, are perfect examples of additional layers of indirection that help us solve problems.

While APIs make this work easier, the approach is more generally applicable. For example, imagine you have an ERP system that is aging and the thought of living through another ERP transition scares you to death, or at least adds one more reason to consider early retirement. Imagine you add a user interface layer between the existing ERP and its users. This could require consuming an API provided by the ERP vendor, wouldn’t that be awesome, or screen scraping or via other less exciting means. When this is complete the new ERP system can be installed and connected to the user interface developed above. The two systems can be brought to a consistent state and the connected user interface can be used to keep them that way. Transaction responses can be compared until you’re confident in the new system. At this point the old ERP system can be retired. You have transitioned to a new ERP system and the users are unaware, that’s success!

There are two main points I think are worth noting. First, an additional layer of abstraction can free an IT organization to make changes without impacting end-users. Second, end-users shouldn’t use the provided user interfaces of institutionally important applications, but rather be provided with screens and applications we develop on top of APIs we control. Installation of a new application is not complete until an API we control is designed and used to create an abstracted user interface that exposes the desired functionality. When applications are installed using this model, they are more easily replaced. Freedom via abstraction!