100th Blog Post: What I know about Software Development and Crisis Management

I started writing this blog post about this…

Opinions on policy and politics aside, this article on the struggles of healthcare.gov tells a classic tale of large software development project failures, and how not to react when trying to solve the issues.

But as I continued to write my thoughts, this became more about my views on software development and crisis management. So, enjoy (or ignore, or comment).

Hopefully, it is worthy of my 100th blog post (which this is).

On software development

  • Software development, no matter how much research is done on development methodology effectiveness, is as much art as it is science. Treat the artists like factory workers, and you get what you deserve.
  • Not all software developers are great at developing all applications in all languages, using all tools. Web development, especially when connecting to legacy systems, is a specific discipline. The best Web developers don’t work at large contract system development houses, in my experience. That’s usually because there are more interesting projects in the world for them to work on.
  • In software, one of the most common reasons for failure is complexity. For large projects like the one cited in the article, you have to work very hard to come up with an architecture that makes concepts and aspects of the application simple to extend and use. This is not cheap—it takes time and significant energy from artists to achieve. But, when it is achieved, it is a beautiful thing. You owe it to yourself to experience this at least once in your career.
  • Never ignore or otherwise skimp on quality User Experience Design (UXD). Never.
  • Designs change. New needs and unidentified technical risks are discovered. Requirements evolve. But if you want to get something done on time, quit changing the (in scope) requirements. If a good design forces you to re-write significant parts of your requirements, that is a good sign that you didn’t write effective requirements. In fact, they may not be true requirements at all. And, BTW, a list of features from a competitor’s brochure are not requirements. If you don’t how to write effective, (mostly) design-neutral requirements, learn. Start here. Then read this.
  • Well-written use cases—that everyone on the team has read and understands—solve a lot of problems. So does identifying the operating environment and performance goals sooner rather than later. Tip: Print this stuff out and hang on the walls were the developers and testers sit. Or (if you want to save some trees) make a JPEG of it and ask everyone to set it as their desktop wallpaper for the duration of the project.
  • Make developers create unit and integration tests for their code. Have them do code reviews with peers. Make them use code comments and consistently format their code. Make them fix their own bugs. Hold them accountable to the quality of software build they provide to the test team. Make them perform design checkpoints with their peers (or even customers) for any significant component. Anything less and they are not a serious professional developer.
  • Developing for security and performance are specific skill sets, but these are most often “want to” issues. As in, if a talented developer truly wants their product to be secure and fast, they will figure out how to make it happen.
  • Do daily stand ups. Make the product manager show up.
  • A good tester is as valuable as a good software developer. So is a good technical documentation person. They are artists too.
  • Give them all good, fast computers with at least two monitors and reliable Internet access. You want something done faster, remove inexpensive barriers.
  • Don’t make them pay for coffee. And don’t buy the cheap stuff.

On crisis management (in IT)…

  • The comment in the article about the “war room” is spot-on. I call it “the body cannon”. When **it hits the fan, there is usually some executive wanting to pull anyone and everyone off their current work and throw them at the problem, with no real plan. They are simply hoping that through volume, that the problem will be solved faster, when in fact it usually has the opposite affect. The better approach: select a small team of talented, trusted artists (that know the code!) and simply ask them: “What do you need?”, and then get it for them ASAP. Then, be prepared to pay the price of pulling these people off their current project (usually in the form of a schedule slip), once the crisis is over. These things are always about choices, not decisions.
  • Yelling at software developers doesn’t work. People that choose a career creating art by typing all day with headphones on are not the type that react well to yelling. If you don’t know how to convey a sense of urgency without yelling, the problem is you. If you don’t understand the problem or software really well, get out of the way.
  • If you have hired talented people, they can become heroes. Let them.

And, finally, if you are making an application for use in healthcare, take it seriously. Lives are at stake. It can still be fun and rewarding, but the problems within healthcare are large and demand our best efforts all the time. Now, go be great.

ACR 2013 – Patient Engagement for Radiology

 

 

 

Presentation by Dr. Alan Kaye (Advanced Radiology Consultants) at ACR 2013 Imaging Informatics Summit, quoting Dr. Rawsson: “It’s hard to put the patient at the center of the universe if you’re sitting there yourself.”

Culture of Patient Engagement

Imaging 3.0 at ACR Annual Imaging Informatics Summit

Quote: “If you don’t like change, you are going to like irrelevance even less.”

Dr. Bibb Allen talking about the importance of accepting change to the practice of Radiology, explained the rationale behind the American College of Radiology’s Imaging 3.0 framework.

Imaging 3.0 - Dr. Bibb Allen

Articles: EHR Stress

The benefits will come, but we must get through the change and this will be painful. Think of the shift from film to filmless, and paper to paperless (with coded, structured records) is this, times a thousand.

Post-SIIM 2013 Annual Meeting Reflections

Another great SIIM annual meeting is behind us and it was great, as always. I am going to post some thoughts and reflections this week.

Today, I have been thinking about analytics and, in particular, the use of a workflow engine and a standardized set of terms and definitions (such as what is being defined in SWIM) to ensure analysis of workflow events (type, timing, relationships, patterns, etc.) consistently across systems.

There were several great talks by Dr. Brad Erickson and Chris Meenan and others on the topic and these were followed by a large turnout of engaged attendees for a SWIM demo (see pic below).

...SWIM lessons
…SWIM lessons

My thoughts…

  • The use of a mature, off-the-shelf (open source or commercial) workflow engine has been considered by PACS and RIS vendors, with some attempting to use them in their product. It has not been widely adopted for two main reasons (I believe)…
  1. Most PACS from large vendors were bought, not built by them—the risk of replacing the built in logic with an external engine without introducing functional regression is high (read as: it would be expensive);
  2. Unless the workflow engine spans several systems, it would not have the full benefit (see more on this below).
  • The workflow examples cited often started with the arrival of the image objects from the modality (initial event that starts the workflow channel). Ideally, the workflow engine extends to before the order is placed, managing the order placement, decision support to ensure the right procedure is ordered, scheduling, protocoloing, and acquisition, along with the reading and post-processing steps. It should also span to the results distribution and archiving, managing the timing and destinations of the report and the lifecycle of the historic imaging data.
  • One of the limitations of using a parallel image management pipeline (e.g. sending images through a system before arriving in PACS) in order to detect the event that triggers the workflow can introduce some points of failure. Consider if the system integrated with the workflow engine goes down and images don’t get to the PACS—this outage would limit the value of the integrated image management and workflow engine system. A possible solution is to extend PACS and other systems, such as the RIS, EMR, CDS, VNA, Enterprise Viewer, document management system, etc. to expose the event information. This would allow the workflow engine to apply the desired workflow rules and orchestrate the data flow and work steps without being a potential bottleneck.

More thoughts from SIIM later. Stay tuned.

Article – Beware: The top 4 hurdles to a successful EHR implementation

Check out this article. Some fairly common observations for an IT veteran, but good advice for EHR buyers.

Some mitigation tips for each point (read the article for the 4 hurdles)…

  1. Build in resiliency. Evaluate options to operate using locally cached data , if supported.
  2. Learn ITIL, and follow the prescribed best practices. If you do, you won’t be putting in upgrades without putting it through a test plan on a test system before moving to production.
  3. If the EMR allows customization of “templates” (or forms), they need to be validated with the representative user communities before imposing them. Some structure, and form element input validation, is needed to ensure completeness and quality of records.
  4. The application and system performance needs to be considered in the overall plan. Inventorying and analyzing transaction and interaction types and volumes, and working with the vendor to spec a system that meets the need, if an important but often overlooked step. Also, assessing the EMR for ease of scalability prior to purchase is recommended.

In regards to the comments on the trade off of lost productivity vs. potential new revenue, check out this post from a month ago.

Quebec EHR …the difference 2 years makes

The news from today (May 2013) “Quebec to expand $1.6 billion EHR“. And, from 24 months ago (May 2011), “Quebec’s EHR late and over budget, AG says“.

One thing is for sure: implementing an EHR of that size and scale (with public funds), is not for the faint of heart.

Apps for Health – Tips for Building an App, Key Trends in Health IT

Prezi presention from @azbib (Heart and Stroke foundation) from today’s Apps for Health event.

Product developers, have a read: 10 great, practical tips on approaching app development that applies to mobile and traditional application products.

Also, some key trends in health for 2013 (originally from Forbes).

Blog – Hospital versus clinic EMRs: what’s the difference?

Technology is easy these days—it really is. Knowing how to use it to solve a problem is harder. And truly understanding the problem is often the difference between success and failure.

I have always felt that one needs to understand the motivations, habits, and even fears of the user, as well as the environment where the product will be used, before design can start.

In this commentary, the author compares the differences between the hospital and clinic environment, and the people working there. I found it insightful, well-written and I learned some new things—check it out.

Article – DoD yanked from health records project

This article is intriguing (and a bit depressing).

First, because it shows once again that the amount of money (say like, US$1 billion) that you throw at a problem does not assure success. Aligning goals and system design principles—and getting firm commitment from all stakeholders—is critical, and it doesn’t seem like that happened here.

Also, there is no mention of the use of commercial HIE technology for record exchange. The article mentions the exploration of commercial EMR technology vs. a custom (“home grown”) EMR, like the VA’s VistA. How is the ONC—a government agency—promoting the use of HIE solutions as part of their patient record evolution, but the VA and DoD not looking at the same approach?

Finally, the vision of an open system is not flawed. And by open, I mean interoperable with modern Web-based APIs. It could even mean open source.