Digital Transformation Building Blocks

This post describes technology building blocks to enable digital transformation at Utah Valley University, but the principles and concepts are universal. Utah Valley University is digitally transforming. The faculty and staff are excited about the possibilities and are envisioning a better future for our students. Students should expect our transformation to yield richer experiences, enhanced learning activities, increased skill acquisition, and a less burdensome path to completion and success. Our faculty and staff should experience less administrative burden allowing them to provide exceptional care, exceptional accountability, and exceptional results to our students and one another.

To facilitate digital transformation, the Office of Information Technology (OIT) and Academic and Student Digital Services (ASDS) must also transform. They must provide reliable and easy-to-use technology solutions that faculty and staff can use to enhance their interactions with others and improve the products and services they deliver. OIT and ASDS must adapt, modernize, provide existing products and services at a reduced cost, and provide new products and services with exceptional customer service.

In general, products and services should be available via self-service, 24 by 7, and ample support to make technology consumers successful, satisfied, and even delighted. The remainder of this post describes architectures, principles, and philosophies intended to make these necessary changes and the dream described above possible.

Application Programming Interfaces

Application programming interfaces (APIs) make it easier for others to interact with applications, create and use alternative user interfaces, and use the available services for alternative and even unexpected purposes. When acquiring or developing an application, it must have an API, and preferably a RESTful one. Data, services, and processes needed to build an application are used to create an API. The resulting API is then consumed to deliver a user experience. Mashup Corporation by Andy Mulholland et al. is a short fictional read that illustrates these concepts and the enabling power of APIs.

To make them most valuable, APIs must be exposed and consumed through API management tools. APIs should be more than simple JSON-based CRUD interfaces; APIs should expose appropriate business logic so that API consumers cannot violate required business processes. Enforcing this principle allows others to build delightful user experiences without institutional concern about policy, practice, or process compliance.

Domain-Driven Design

Everyone should read at least the first two chapters of the book Implementing Domain-Driven Design by Vaughn Vernon! Here is my super-short summary of domain-driven design (DDD) and those chapters — bring domain experts and developers together to create a ubiquitous language embedded in the application code itself. In addition, define or determine bounded contexts wherein this language is valid. This exercise helps software developers genuinely understand the business processes they are being asked to automate. It also helps the business participants understand the code being written and allows them to question decisions, test assumptions, and find bugs before deployment. This collaborative group of business leaders and developers is “the team”; success or failure is in their hands.

Microservices

Microservices are an architectural style that will be used at UVU to create larger systems. Systems built using microservices are loosely coupled; I would even go as far as saying they are highly decoupled, implement a single business capability, have well-defined interfaces, and communicate using only these interfaces. The size of a microservice is governed by the associated bounded context, go, and read the DDD book! At UVU, an essential part of a microservices’ interface is its ability to raise events. Go figure out why.

Event-Driven Architecture (EDA)

Systems that poll are inefficient! Build systems that raise events so other systems don’t have to waste time and resources. You can keep asking me if you must do this, but you can be assured that when I change my mind, I’ll let you know. If you didn’t find the humor in the last sentence, then reread the links.

Application Acquisition

When we purchase applications, we should give preference and a strong preference to those running in the cloud. When we build services or applications, they should use the most abstract service offerings that make sense. In other words, we should not instantiate servers and consume storage and then build queues, notification services, etc. We should instead use services such as queues, notification systems, serverless functions, etc.

DevOps

DevOps is a culture and practice that we hope will result in rapid development, testing, and software deployment. We also hope this increases accountability by allowing those who develop an application to be responsible for running and supporting it. Nothing motivates a developer to fix a bug more than to wake them to fix it repeatedly. Teams, DDD teams, are in charge and responsible for the functionality, performance, and reliability of “their” products.

If those in the hardware world think you’re off the hook, think again. Software is eating the world. The days of interacting with network switches, routers, firewalls, servers, storage appliances, AV equipment, etc., are over. Learn to program, learn to configure hardware devices using programs, remember to use DevOps principles to configure, test, and deploy hardware platforms as rapidly as “other” developers – that’s right, you just became developers!

Where to Compute

We built data centers and populated them with servers, storage systems, and network components in the past. As CPU performance increased, computers became more able to run multiple applications, but stability due to unintentional application interaction made this approach intolerable.

We found ourselves with many underutilized servers running single applications to maintain reliability. Along came server virtualization enabling us to instantiate multiple virtual servers on each physical server. Over the past several years, the number of physical servers has diminished considerably.

Well, we’re in the middle of another paradigm shift. We are continuing our journey that will result in our compute and storage being somewhere else. Acquired applications will also run in the “cloud.” in either case, they will not be housed here.

Networks

Unlike server and storage, I believe we will have a wired and wireless network on campus for the foreseeable future. However, the way we deploy, configure, and maintain these networks will change drastically. Remember, software is eating the world, and networking is not an exception to the rule. Network components will be physically installed in some generic way and then configured remotely via software. In a DevOps fashion, when a problem occurs, you’ll figure out what went wrong in the configuration script, you’ll repair the script, you’ll test the script, and you’ll redeploy.

Final Thoughts – For Now!

We have a great team! Let’s pursue all this FUN with great enthusiasm. Let us share our best thinking with others: share code on GitHub, answer questions on StackOverflow, blog about your experiences, publish papers, present at conferences, participate in panel discussions. In short, learn, teach one another, and teach the world!

TAN: Trailer Area Network

Introduction

A friend of mine, Scott Lemon founder and CTO of Wovyn, and I were discussing my travel trailer API and he coined the term, trailer area network or TAN. My TAN is going to be based on WiFi because of the ubiquitous availability of WiFi connected devices including microcontrollers, commercially available connected devices, and mobile phones and tablets. To this network I’ll attach microcontrollers that control trailer subsystems. These subsystems will connect via WiFi to a trailer hub that will expose subsystem functionality through the travel trailer API, see Figure below.

In this figure Internet connectivity is acquired through a mobile hotspot with the help of a cell signal booster. The mobile hotspot allows 15 devices to connect to it. One of the 15 devices will be the TAN Hub. The TAN Hub will create a subnet that subsystems will attach to, there could be many. Communications between the subsystems and the TAN Hub may use various protocols, but the TAN Hub will expose their functionality through a REST based API.

Components

There are three main sets of components that are needed to create the described TAN. For Internet connectivity we need a signal booster and a mobile hotspot. We need a server of sorts to act as the TAN Hub and a WiFi connected microcontroller unit to use in the construction of subsystems.

Internet Connectivity

I have selected the AT&T Unite Explore, also known as the Netgear AC815S, device as my mobile hotspot device. It seems to get good reviews and is reasonably priced. I may experiment with other hotspots in the future.

I acquired a weBoot Drive 4G-M Cell Phone Signal Booster to facilitate the acquisition of cell signals in remote locations. In the future I may experiment with various antenna configurations to increase the usable range.

Trailer Area Network Hub

For the TAN Hub I will use the Corewind WiFiG25 microcontroller module described in my original post describing my trailer API. This module is based on a 400 MHz ARM processor, has ample memory, and built in WiFi and runs the Linux operating system.

I expect to add an API management system to this module and communication systems to enable it to connect to and communicate with trailer subsystems.

Subsystem Controllers

There are many potential trailer subsystems that can be exposed and interacted with through an API. Examples include: leveling jacks, stabilizers, tank levels, battery levels, solar energy generation, awning, pop-outs, security, weather data, etc. Each of these will inevitably require special sensors, actuators, etc., but I am hoping to standardize on the microcontroller modules used to control the subsystems and communicate to the TAN Hub.

I will be experimenting with the ESP 12-F module based on the ESP8266 chip. This is an impressive module featuring a 32 bit MCU, integrated WiFi, tcp/ip stack, 4 MBytes of Flash memory for user programs, and all this for $1.74. It is capable of communicating with other hardware using SPI, I2C, Serial and other low level communication protocols.

These communication channels, as well as the available GPIO lines, will be used to interface these modules to subsystem specialty hardware such as switches, sensors, etc. The integrated WiFi will be used to connect these subsystems to the TAN Hub.

Summary

Future posts will describe various details related to the programming, configuration, and use of the components discussed in this post. There is much to learn, much to do, and a lot of fun to be had.

Travel Trailer API

Introduction

I have recently purchased a travel trailer and desire to instrument and automate it. I want to control and monitor all of its systems using my iPhone or other mobile devices. When I am out in the sticks (defined by my son to be wherever there is no Internet connectivity) I want to connect to the trailer’s systems via a local WiFi network. When the trailer is in a location where LTE, or other data services are available, I want to interact with the trailer via the Internet.

While there are many systems to monitor and control, there are two fundamental technological pieces they all require. First, a WiFi network must be provided for mobile phones and other devices to connect. Second, a central controller is needed to manage the myriad of systems that will be connected to this network and manage the associated API.

Rather than boiling the ocean, I intend on focusing on one simple representative system, a weather station, and connecting it to our trailer. Success with this system requires solving many issues that will simplify the inclusion of additional systems in the future. The rest of this post will describe a path forward. A series of future posts will describe the details associated with each subsystem.

System Description

The goal of this project is to enable a mobile phone user to view weather data acquired from a weather station connected to our travel trailer. There are a number of modules or subsystems required to accomplish this:

  1. A mobile phone that connects to a WiFi network and an application that consumes an API that returns weather station data.
  2. A secure WiFi network or trailer area network (TAN). While security of weather station data may not be required, future projects will require it and we should design with security in mind.
  3. A central controller that connects to the provided TAN and exposes an API management tool. This will enable mobile phones and other devices to access available data in a secure and managed way.
  4. A 433 MHz receiver connected to a microcontroller based module will convert the signal acquired from the weather station into an appropriate API.
  5. A weather station that transmits collected data on the 433 MHz band.

The weather station will collect environmental data and transmit it over the 433 MHz radio frequency band. This signal will be acquired using a 433 MHz receiver that converts this analog signal to digital for consumption by a microcontroller based system. The digital signal will be parsed appropriately and exposed through an API. This API will be managed by the central API management system and exposed to other devices over trailer area network. The mobile phone will consume this API and an appropriate mobile phone app will present it to the user.

Chosen Components

There are many possible components to choose from to accomplish the goal and steps outlined above. The components I’ve chosen and describe below are simply representative of the myriad of possible choices.

Weather Station

I will use the AcuRite 5 in 1 Professional Weather Center model 01536. This product is compact, easily mounted on a trailer and broadcasts weather data on the 433 MHz band. The outdoor component measures the temperature, humidity, rainfall, wind speed and wind direction. This data is combined into a packet and broadcast periodically. The indoor component is a nice color display that displays this data and archives previously received data. The indoor and outdoor components are paired through a simple channel selection mechanism.

433 MHz Receiver

I acquired the Sunkee 433 MHz superheterodyne wireless receiver module to convert the analog signal broadcast from the weather station into a digital signal appropriate for consumption by a microcontroller based system. This module can operate from roughly 3 to 5 Volts and requires a simple wire antenna. It outputs a digital signal easily read by standard microcontrollers.

Microcontroller System

I have chosen to use the Corewind WiFiG25 microcontroller module. It has a 400 MHz ARM processor, plenty of memory, built in WiFi connectivity, numerous I/O lines of various types, runs the Linux operating system, supports many programming languages, and only costs about $30.

Software Components

With the receiver connected to the microcontroller module, the rest is up to the software. A simple program will be written to acquire the digital I/O data from the receiver and store it in memory. The Kong API management tool will be implemented on the microcontroller to expose the collected weather data via an API. This API can then be consumed over the WiFi network by a mobile phone and associated application and our task will be complete, for now.

Summary

The components described above, and a few others, will be combined to expose weather data via an API for consumption by mobile phones and potentially other devices. The next post will describe the setup and use of the receiver module and a description of the data collected from the weather station. A series of posts will follow describing the remaining steps to accomplishing our goal.

Indie Educational Technology

Introduction

Last week I spent some time at Davidson College discussing Personal APIs and Indie Educational Technology with faculty, students, and staff from several other institutions of higher education and commercial entities, Known and Reclaim Hosting, that facilitate this work. This was a fantastic gathering of bright people and I can’t wait to be with them again. Thanks to Kristen Eshleman for getting us together, to Ben Werdmuller and Erin Richey for their instruction, to Audrey Watters for her insightful description of indie, and to Tim Owens, Jim Groom, Kin LanePhil Windley and Troy Martin for always making me think better. After returning to my day job I found myself asking the question, what is “indie educational technology”?

According to CNN “If it’s cool, creative and different, it’s indie” and the Urban Dictionary defines indie as, “an obscure form of rock [music] which you only learn about from someone slightly more hip than yourself.” I had to travel to North Carolina to have people, more hip than myself, educate me! I thank them and hope they continue to help me progress.

While I’m pretty sure the term “indie” in this context meant “independent of vendors and personal”, for my purposes I’m going to define indie educational technology as information technology that is cool, creative and different used to enhance the educational process.

Indie technology benefits both students and educational institutions. Students have a greater sense of ownership and motivation when they (a) control their personal information and (b) are able to interact with institutions with both institutionally provided applications and alternative systems (McCombs, 1997). Institutions benefit by having alternative application-hosting options and are not unnecessarily burdened by housing personal student information with its associated liability.

For example, let’s consider a traditional, non-indie, university registration system. One or more centralized systems contain university information about courses, classrooms, and instructors. In addition to this information, these systems contain personal student information that students are required to submit to participate in the registration process. To register for classes, students present user credentials to the university provided registration system, register for classes, and end their interaction. In this model the university retains personal student information, insists that students use the university provided system, and refuses to make alternative systems possible or feasible.

Consider an alternative, the indie approach. Like the non-indie system described above, one or more centralized systems contain university information about courses, classrooms, and instructors. However, personal information is not retained in this system. A student’s personal information is housed in a student-controlled system. Students authorize the university’s system to access necessary information. In this case students register for classes by presenting credentials to a registration application of their choice. This registration system requests and receives authorization from the university and the student to acquire university and personal student information, respectively. They register for classes and end their interaction. In this model the registration application disposes of personal and university information it was exposed to, while the university system retains the information necessary to indicate successful registration. In this scenario students may choose to use the university provided registration tool or alternative systems. In addition, the university does not house personal information or bear the associated liability. That’s cool, that’s indie!

So what makes this technology cooler, more creative, and different from what currently exists? Let me suggest that it is because indie technology will have several characteristics:

  • Personal API Enabled
  • Substitutable
  • Open Source
  • Modular
  • API-Based
  • Event-Driven

Let me elaborate on each.

Personal API Enabled

A personal API (PAPI) is an interface to personal information and resources. The resource owner protects these resources through explicit authorization. There are at least three key benefits of developing and using a PAPI:

  1. A PAPI changes the expectations of users. They develop a sense of ownership of their information and resources and begin to expect institutions to respect their rights and privacy. The use of a PAPI at an institution of higher education yields a perfect opportunity to educate students about these issues and help them understand what they should expect from other vendors and providers. They benefit from the ability to disassociate from institutions by simply revoking authorization to their data.
  2. A PAPI eliminates the need for a single university or vendor-provided application that all users must interact with. Users interact with their PAPI using applications of their choice. Institutional systems request permission to access personal information through the PAPI to perform needed functions. Institutions may provide applications for users, but the PAPI facilitates the creation of alternatives.
  3. A PAPI frees institutional technology modules from having to unnecessarily hold personal information. Institutional systems request needed personal information through the PAPI to perform their functions. Institutions should find this attractive because they will house less personal information, reducing their liability.

Note that giving people a personal API and letting them control their data, doesn’t mean that they get to control the university’s data. A PAPI lets people control the data that is theirs. For example, their phone number is their data. Their grades, on the other hand belong to the University. In addition, if students exercise their right to not authorize university access to needed personal information, the university is not obligated to fulfill the desired student request. University policy and process must still be followed.

The institutional complement to the PAPI is what we call the University API (UAPI). Through the UAPI an institution protects its resources through explicit authorization. In our example indie class registration system, the UAPI would make course, classroom, and instructor information available to the student chosen registration application. In addition, the UAPI would record the necessary registration outcomes.

Substitutability

Substitutability is the ability to use alternative systems or services to accomplish specific functions and move from one platform to another with ease and at little expense. This is applicable to both users and institutions.

Substitutability benefits users by allowing them to move their systems and services to alternative providers. They are also free to choose alternative applications to perform functions of interest. Institutions should facilitate both by pursuing strategies that allow authorized access to institutional information and consume necessary personal information through a PAPI respecting the user’s expectations of privacy.

Institutions benefit as well. Their systems can be operated on multiple platforms and through the use of technologies such as the UAPI and the PAPI, alternative systems can be used to accomplish institutional functions. If institutional systems only work properly when hosted at a single provider or moving them is onerous, institutions leave themselves vulnerable and open to the policies and practices of that provider. The inability to easily substitute one provider for another brings us back to our current state of affairs.

Open Source

Created systems and services should be freely available to others. First, this is what the cool kids do – indie. Second, by making them and API definitions freely available, others are more likely to adopt the technologies. Wide adoption results in many smart, hip people working on the same problems, resulting in better solutions. Licensing them appropriately protects our ability to use the things we develop.

Modularity

Modularity facilitates and drives an increased pace of innovation. Each module should deliver a small set of functions within a single bounded context as defined in the domain-driven design process. While these modules can be created using various techniques, at Brigham Young University (BYU) we will be defining them as microservices. These microservices will result in stand-alone modules that are easily understood by developers and will encourage extremely loose-coupling, facilitating a building block mentality to building systems. This approach will drive innovation in the core processes of BYU. 

API-Based

Each module will have an API that enables communication to and from the module. The API simplifies the use of the module and abstracts away the internal implementation. This abstraction permits changing the underlying implementation while protecting systems that rely on the module’s API. 

Event-Driven

While not strictly necessary, event-driven architectures are more efficient and absolutely cooler than polling-based systems. I think this alone makes event-driven, modular design a part of indie technology!

In a polling-based system you only become aware of changes when you ask if changes have been made. For example, in a registration system you determine how many students have registered for a particular class by asking (polling) the system. In an event-driven architecture, each time a student registers for a class an event reflecting this activity is posted to interested listeners. This results in more efficient communication and more timely responses to change. What could be more indie?

Now What

At Brigham Young University we intend on building many, if not all, of our core academic systems and services using modules with the above characteristics. The result will be a collection of modules that perform core functions of the institution, but are likely usable by others.

I hope that we can find ways of including others outside of the BYU community in the creation of our functional modules, systems, and services. Including others will make our work better, but more importantly will result in definitions and implementations that are more generic, enabling others to use them more easily. Each module, system, and service will have the characteristics outlined above making their use elsewhere practical and possible.

Finally, I hope we can all find a way to meet regularly to showcase our attempts, failures, and successes. We, at BYU, are open to conferences, workshops, or other venues where we can all continue this discussion.

myDoorbell: High Level Design

Introduction

In my previous post I described the resources available in the wiring box where myDoorbell will be installed. As illustrated in the following figure, the available resources consist of a 16 Volt AC source and a connection to a doorbell button with an integrated lamp.

When the button is pressed, approximately 1 Ampere of current flows through the circuit. This current at 16 Volts AC yields 16 Watts of available power (16 Volts * 1 Ampere). This is sufficient for anything I am envisioning. For a more thorough, yet simple, explanation of Ohm’s Law click here.

Features and Hindsight

As pointed out in my first post on this subject, myDoorbell will have several unique characteristics or features:

  • Play ringtones uploaded by the user to celebrate seasons, holidays, birthdays, etc.
  • Record the time and date of each ring
  • Be configurable
  • Send a text message indicating a ring, if configured to do so
  • Be silent if configured not to make noise when babies are sleeping, pets shouldn’t be disturbed, or the owner just doesn’t want to know visitors are present
  • Access other Web resources such as APIs, webhooks, etc. 

As the saying goes, hindsight is 20/20. If I were designing myDoorbell all over again, I actually am, the next thing I would do is design the API that would describe the HTTP resources, methods, and error codes used to implement the above features and behaviors. However, when I began this adventure I didn’t know much about APIs, HTTP, and related technologies. Recall that this lack of knowledge was precisely why I started this adventure.

I will proceed in roughly the order I actually took, but want to encourage any readers creating hardware or software systems that they should design the API first. Since software is eating the world it is clear that any real system will contain software, and I believe that all such systems should have an API. Creating the API first will result in a better product and will reduce the risk of cresting hardware and software components that in the end are unnecessary.

Necessary Components 

Several hardware and software components are necessary to implement the characteristics and features listed above:

  • myDoorbell needs Internet connectivity to facilitate the uploading of ringtones, to enable the calling of other APIs or to raise events, and allow the acquisition of configuration information and updates. Since there is no wired network connection in the doorbell wiring box, this connectivity will be provided by a WiFi module. 
  • A microcontroller is needed to interact with the WiFi module, provide a compute platform for the web server implementing the myDoorbell API, interface with a nonvolatile memory, and enable the detection of the state of the doorbell button. 
  • A software module is needed that time slices between detecting the state of the doorbell button and dealing with incoming network traffic. It must also copy incoming ringtone data to the nonvolatile memory. Finally, this module must implement outgoing requests and the raising of events.
  • A simple hardware module is needed to provide doorbell button state information to the processor.
  • A nonvolatile memory is needed to store ringtones and configuration data. This memory needs to be writable by one microcontroller and readable by another, two ported.
  • A second microcontroller is needed to copy ringtone data from the nonvolatile memory to the audio system. 
  • An audio subsystem is necessary that converts mp3 encoded music to audio and amplifies the resulting audio to a sufficient amplitude to drive a speaker to the required volume.
  • A power supply is needed to convert the 16 Volt AC to a 3.3 – 5 Volt DC source for the digital electronics and a higher DC source for the audio amplifier. 

The following figure is a block diagram illustrating the hardware components listed above. Note that there are two modules for detecting the doorbell button state because most commercially available doorbells have the capability of ringing for both a front and rear door.

There are of course many ways to design any system, and I am not arguing that this is the best or even a reasonable design. In addition, as technology changes so do the choices and possible optimizations. My intent was to learn, and I learned a lot. In my next post I will describe the power supply module. This seems simple, but hang on and keep your arms and hands in the ride at all times!

myCoreDump

Introduction

I hope you enjoy this core dump! The thoughts are so interrelated and connected it is difficult to optimize the presentation so you may need to apply your own defragmentation to get it. In addition, the order is not intended to indicate priority, it is all freaking important!

University API (UAPI)

When acquiring or developing an application it must have an API, and preferably a RESTful one. If the function of the application is core to university business then it should be exposed through the UAPI. If it is not a core function of most, if not all, educational institutions, we should expose the API through our API management tools, but it shouldn’t be part of the UAPI.

Personal API (PAPI)

When we build a system that will store personal / individual information we should consider how we might leave the information in the hands or possession of the individual and access it for our use through their personal API. Since no one yet has a personal API, for the time being we must provide that as well. This will require you to stretch your imagination and creativity, but that’s good for you.

Domain-Driven Design

Everyone should read at least the first two chapters of the book Implementing Domain-Driven Design by Vaughn Vernon! The super short summary – bring domain experts and developers together to create a ubiquitous language that is embedded in the code itself. In addition, define or determine bounded contexts wherein this language is valid. Without this you won’t understand how we’re going to build solutions and you won’t have a clue what is in and what is not in a microservice. Read it!

Microservices

Microservices are an architectural style that will be used at BYU to create larger systems. Systems built using microservices are loosely coupled, I would even go as far as saying they are highly decoupled, they implement a single business capability, they have well defined interfaces, and communicate using only these interfaces. The size of a microservice is governed by the size of the associated bounded context, go and read the DDD book! At BYU an important part of a microservices’ interface is its ability to raise events. Go figure out why.

Event-Driven Architecture (EDA)

Systems that poll are inefficient! Build systems that raise events so other systems don’t have to waste time and resources. You can keep asking me if you have to do this, but you can be assured that when I change my mind I’ll let you know. If you didn’t find the humor in the last sentence then go read the links again.

Application Acquisition

When we purchase applications we should give preference, strong preference, to those that run in the cloud. In fact, before we choose an application that is not available as a service choose someone in your group you don’t love and care about to come get my approval.

When we build services or applications they will run at Amazon and use the most abstract service offerings that make sense. In other words, we should not instantiate EC2 servers and S3 storage and then build queues, notification services, etc., but instead should use services such as SQS, SNS, Lambda Functions, etc.

DevOps

DevOps is a culture and practice that we hope will result in the rapid development, testing, and deployment of software. We are measuring the number of deployments / week, failures / week, and time to recovery. We are promoting small changes, thorough automated testing, and deployment to production often. Your team (the DDD team) is in charge and responsible for the functionality, performance, and reliability of “your” product. 

If those in the hardware world think you’re off the hook, think again. Software is eating the world, software is eating your world. The days of interacting with network switches, routers, firewalls, etc. are over. Learn to program, learn to configure hardware devices using programs, learn to use DevOps to configure, test, and deploy hardware platforms as rapidly as “other” developers – that’s right, you just became developers!

Where to Compute

In the past we built data centers and populated them with servers, storage systems, and network components. As CPU performance increased computers became more able to run multiple applications, but stability due to unintentional application interaction made this approach intolerable.

We found ourselves with many underutilized servers running single applications to maintain reliability. Along came server virtualization enabling us to instantiate multiple virtual servers on each physical server. Over the past several years the number of physical servers has diminished considerably.

Well, it is time for another paradigm shift. We are now embarking on a journey that will result in our compute and storage being somewhere else. We will take advantage of Amazon to deliver what our applications and services need to run. Acquired applications will also run in the “cloud”. in either case they will not be housed here. Resources used previously to purchase servers and storage, and maintain them will be redirected to this new endeavor.

Networks

Unlike server and storage, I believe we will have a wired and wireless network on campus for the foreseeable future. However, the way we deploy, configure and maintain these networks will change drastically. Remember, software is eating the world and networking is not an exception to the rule. Network components will be physically installed in some generic way and then configured remotely via software.

In a DevOps fashion, when a problem occurs you figure out what went wrong in the configuration script, you repair the script, you test the script, and you redeploy. Remember, we’ll be watching how often you deploy, how many failures occur, and how long it takes to recover.

The days of hugging these devices are over. If you want one to hug, you can have one of the old ones and keep it in your office – disconnected from the network of course.

Domain of Ones Own (DoOO)

As we embark on this new path it is a great time for you to consider contributing to the content of the Internet. Let your light so shine by getting a domain of your own and sharing your goodness and skills with others. get one at domains.byu.edu. Here you can blog your greatest thoughts, post content that you syndicate to Facebook, Twitter or other services. Go learn, learning is fun!

We are offering this service to all students because we believe they should understand more about how the Internet works. We believe they have much to offer the world and they need to know they can share it with little help from service providers. What they build is transportable to other hosting services and is theirs! In the future a DoOO will enable an individual to have a portfolio and expose this and much more through their personal API (PAPI).

Final Thoughts – For Now!

We have a great team! Let’s pursue all of this FUN with the greatest enthusiasm and Heaven will shine down on us. Let us share our best thinking with others: share code on github, answer questions on stackoverflow, blog about your experiences, publish papers, present at conferences, participate on panels. In short, learn, teach one another, and teach the world!

myDoorbell: A Learning Adventure

Introduction

After being a university chief information officer (CIO) for more than a decade, I decided to refresh my technical skills acquired through formal education and practice as an electrical engineer. I learn best by doing, so I picked a project I was interested in pursuing with the end goal being the learning, and not the finished product. I intend to share several posts that I hope illustrate the things learned and hope they are of value to the reader.

My Project

I have interest in the Internet of Things (IoT) movement and wanted to make strides towards making this practical, simple, and secure. I believe connected devices should be simple and consume little power. This likely requires devices that wake periodically, connect to some sort of network, and then go back to a low power state. After some experimentation it was clear, at the time, that WiFi was a real power hog and wasn’t a likely candidate. However, this realization led me to believe that another router, hub, or coordinator device would be necessary. I recall the effort required to convince homeowners to acquire WiFi routers and looked for an approach that would make this palatable.

I decided the answer was to create a product that homeowners would want to purchase because it excited them, and by the way it contained a network router / coordinator. Once acquired on its own merits, the product would facilitate the inexpensive and simple acquisition of other devices that connect to it. Products worth considering would be interesting to households and would connect to household power:

  • Lamps
  • Televisions or other audio / video (AV) equipment
  • Thermostats
  • Doorbells

Lamps seem simple and boring. However, after implementing my first choice, I know I should have chosen a lamp because it would have been boring, simple, and done! I decided embedding anything in televisions or other AV equipment would require skills and resources I didn’t have. Nest took the thermostat direction and while I disagree on the approach of putting so much technology in a tightly coupled system, I didn’t want everyone to judge my work against a commercially available product. I chose to implement a doorbell because they are ubiquitous, simple and meet my requirements:

  • They are in nearly every U.S. household.
  • Power is available where the indoor ringer is found.
  • They do one thing and no one cares if they do anything else.
  • They are in a good physical location for a network router.
  • They are out of the way, aren’t moved, never unplugged, or inadvertently reconfigured.

I chose to create a doorbell that would function as a replacement doorbell, would act as an IoT network router, and connect this network to the Internet by also connecting to an existing WiFi network. A quick trip to Home Depot revealed that an inexpensive doorbell cost about $13. Even with no experience in product development, I knew I wasn’t going to be able to build a doorbell that also acted as an IoT to WiFi gateway for $13. To be compelling enough to get households to acquire my doorbell it would have to be feature rich:

  • This doorbell would play ringtones uploaded by the user to celebrate seasons, holidays, birthdays, etc.
  • Each time someone rings this doorbell the time and date should be logged.
  • The owner can configure the bell to text them when someone rings.
  • The bell should be easily configured not to make noise when babies are sleeping, pets shouldn’t be disturbed, or the owner just doesn’t want to know you’re there.
  • When the bell is rung it should be configurable to access other Web resources such as APIs, webhooks, etc.
  • The system should be controlled and configured using a mobile app.
  • The doorbell must be a simple replacement of the original doorbell ringer.

While these features increase the likelihood of making it compelling enough to overcome the necessary price point, it certainly eliminates any chance of it being simple.

Summary

In this post I declared my intent to refresh my technical skills through the development of an IoT product, an amazing doorbell, myDoorbell. In the next few posts I will describe how a typical doorbell works, illustrate the general system layout for this new doorbell, describe how to create it so it fits into existing doorbell systems, and discuss many details of the techniques and technologies that make this possible. It will be a fun journey with many twists and turns, but that’s how learning happens!

Domains, Personal APIs, and Portfolios

Introduction

In addition to the traditional educational experience students at Brigham Young University receive, we want them to acquire skills, techniques, and tools that facilitate their current and future learning. We believe students should learn how to control and own their digital identity, content, and personal data. With this goal in mind we have initiated a pilot program using a concept known as Domain of Ones Own. We hope to accomplish several goals using this concept and associated training:

  1. Teach students, faculty, and staff why they should care about owning, controlling, and appropriately sharing their online identity, the content that defines them, and their personal information.
  2. Help individuals understand how to choose a domain name that accurately and professionally represents them to others.
  3. Encourage members of our community to not simply consume, but contribute to the body of knowledge through the use of blogs and social media.
  4. Support individuals in publishing a Personal API (i.e. api.example.com) that allows the owner to authorize others to interact with their personal information and revoke access privileges as desired.
  5. Support students and faculty in creating a portfolio (i.e. api.example.com/portfolio) as part of their Personal API that is owned and maintained by the individual owner, and yet enables the owner to authorize others to consume, contribute to, and evaluate their collection.

Domain of Ones Own

Many members of our community share their pictures, memories, thoughts, insights, and writings on social media sites that are controlled by others. The privacy policies of these sites change over time, access privileges may change, copyright ownership is a concern, and the look and feel desired by the content owner may change without their knowledge, input, or control. Contributors have no control over the amount or type of advertising placed around or even over their content. In many cases they may not be able to easily move their content to other providers, remove content they no longer wish to share, or even pass ownership onto others as desired. We want members of the BYU community to understand that there is a better way.

Consequently, we have chosen to use and teach a concept known as Domain of Ones Own. We first herd about Domain of Ones Own from Jim Groom when he was at the University of Mary Washington. After a visit we were hooked on the idea of freeing our community and using the tool to rethink content ownership, Personal APIs, portfolios, and Learning Management Systems.

Our implementation of a Domain of Ones Own consists of a simple hosted server configured using cPanel and pointed to by the end-user’s chosen domain. We are using the service and tools provided by Reclaim Hosting who provides the tools, hosting, and the process for acquiring domains. With the default, initial configuration domain owners have a blog driven by the Known blogging tool. While this is a great introduction that allows domain owners to contribute immediately, the system is open and can grow as the domain owner’s sophistication increases. The system allows users to set up subdomains, email servers, database servers, and install and run many LAMP stack based applications. The tools and services have been chosen carefully to allow users to move their domain and associated content to other providers easily. Tools were chosen to be immediately useful, provide future flexibility, and help users learn introductory system administration skills that are critical to understanding the world they are in and will inherit.

Domains

We believe every individual should own and control their domain. Choosing an appropriate domain is important. In many cases the domain will be used in a professional capacity for years, perhaps for life. We are creating instructional material, including short video segments, which will give advice on how to choose well. We intend to create these materials in a way that minimizes branding and IP protection so others can easily use them for similar purposes at their institutions.

Personal API and Portfolios

Imagine a world where other sites on the web don’t hold your personal data, but instead request access to the data they need through your Personal API. Perhaps you grant them access to only the portions they actually need and restrict them from others. They use the resources they’ve been authorized to access, perform the business functions you desire, return results, and their access is revoked.

For example, imagine you work for weLovePrivacy.com and it’s payday. The payroll system springs to life and determines how much you should be paid this month. However, it needs to know how much should be withheld for taxes, how much pretax contributions to make, where these should be made, where you want your money deposited, etc. In a traditional system all of this information is centrally held. This centrally held information compels the institution to create systems to enable you to manipulate it, and makes the company liable for any loss of this data. On the other hand, you are depending on the institution safeguarding your personal information and not using it for nefarious purposes, a dangerous assumption.

However, there is a better way. Imagine the payroll system interacts with your Personal API to obtain your social security number, the number of exemptions you are declaring, the name of your 401k vendor, 401k account number, your checking account provider and account number, etc. The institutional system does the computation and disbursements, and your Personal API revokes access to these resources until the next time they are needed. While the institution could store the collected information it may not be in their best interest to do so and could even be released to them with the understanding it is to be used for the sole purpose disclosed to the user.

While it may be a while before ERP administrators are comfortable getting employee data from their personal API, there are plenty of other scenarios where a personal API is useful. Portfolios is an example of such a scenario. An instructor at an institution requests authorization to place assignments into your Personal Portfolio, their request is granted, and the assignments are deposited. You perform learning activities that generate solutions to the assignment, and deposit these in your portfolio. You have authorized the instructor to see them and place their critiques back into your portfolio. Since this is your portfolio it moves with you from one part of your life to another, from one institution to another, etc. It is yours to use and share as you choose.

Summary

It is time for learners to take control of their content, artifacts of education, and personal information. Our desire and intent is to teach these principles to our community and give them the necessary tools. We hope to do so in a way that others can easily use and benefit from.

Freedom via Abstraction

At Brigham Young University (BYU) we have been developing a University API to expose the functionality of a reasonably generic educational institution, while consuming a very specific set of underlying technologies. Our generic institution has instructors, students, courses, classes, and locations. These resources and available HTTP methods are being combined to expose acceptable business processes such as registration, adding and dropping classes, etc. We will continue to add resources and appropriate business processes as necessary to meet our institutional needs.

Our intent is to develop future applications by consuming the University API and will encourage others to do the same. We will no longer consume the user interfaces or APIs of underlying systems. This layer of abstraction will enable us to replace the underlying technologies with new technologies that provide similar functionality. Regardless of the tools or technologies used, those consuming the University API will be unaware of the underlying change. This will give the IT organization the freedom to make changes to reduce cost, modularize monolithic applications, move to microservices, etc. without impacting application developers or end users. This will bring them freedom via abstraction.

I’m writing about this today because in my mind this is an important general architectural pattern that should be followed more often. David Wheeler­­­, a British computer scientist, is credited with saying, “All problems in computer science can be solved by another level of indirection, except of course for the problem of too many indirections.” While most often quoted by programmers in discussions about pointers and similar constructs, I think abstraction layers, like the one discussed above, are perfect examples of additional layers of indirection that help us solve problems.

While APIs make this work easier, the approach is more generally applicable. For example, imagine you have an ERP system that is aging and the thought of living through another ERP transition scares you to death, or at least adds one more reason to consider early retirement. Imagine you add a user interface layer between the existing ERP and its users. This could require consuming an API provided by the ERP vendor, wouldn’t that be awesome, or screen scraping or via other less exciting means. When this is complete the new ERP system can be installed and connected to the user interface developed above. The two systems can be brought to a consistent state and the connected user interface can be used to keep them that way. Transaction responses can be compared until you’re confident in the new system. At this point the old ERP system can be retired. You have transitioned to a new ERP system and the users are unaware, that’s success!

There are two main points I think are worth noting. First, an additional layer of abstraction can free an IT organization to make changes without impacting end-users. Second, end-users shouldn’t use the provided user interfaces of institutionally important applications, but rather be provided with screens and applications we develop on top of APIs we control. Installation of a new application is not complete until an API we control is designed and used to create an abstracted user interface that exposes the desired functionality. When applications are installed using this model, they are more easily replaced. Freedom via abstraction!

css.php