Developing Accurate, Complete and Current Information is a BAD Idea

“Every single decision EOC responders make depends on accurate, complete and current situational awareness.”

This quote comes from a seasoned emergency manager in a recent Emergency Management Magazine article.  Simply said, I don't agree with this key point.  This kind of unilateral thinking leads us down a very dangerous path as it builds up false expectations and breads unrealistic thinking. To quote a colleague, "the mythical quest for perfect authoritative data can be paralyzing."  

"Accurate, complete, and current" information is a nice goal, but is entirely impractical and unrealistic in reality.  In a recent email listserv conversation regarding the Nepal earthquake, a number of very experienced information managers discussed the difficulty in simply keeping up with the flow of information during a disaster.  Perhaps this can be better achieved in the future, but in current practice it is near impossible to manage and achieve, even before a disaster. 

The more important aspect of this is to understand HOW accurate, complete and current the information is. For example, if you know information is two hours old, you can assess the value of it for your own decision making. When you receive information, you should know who it is from, how old it is, and what is addresses.  This "meta-information" is critical for good SA/COP and is actually quite informative for a decision maker.  Granted the information is not ideal, but one can make a more educated decision based on his or her own assessment of the data or information.  

Plus, you  can spend an inordinate amount of time trying to gather accurate, complete and current information.  This can produce a effort-to-outcome imbalance where you spend more time gathering and organizing less valuable information.  Time can be better spent on working with the  information you have, regardless of its condition, which can be more valuable for decision making and action taking during a disaster. 

Also, while I agree with the author's points about thinking through the process for gathering and disseminating SA, the existing processes for developing information requirements are equivalent to throwing a dart at a dartboard blindfolded. While you were pointed in the right direction to begin with, you still have no idea where you are aiming. The result is a set of information put into SA/COP, but not understanding of its relative value across responders and different disasters.

In fact, the decision approach to identifying information requirements gives a false sense of security as you only define information requires you may face and can identify ahead of time. There are usually a host of unanticipated critical decisions the need to be made during a disaster. Developing information requirements is arguably more important for the unanticipated decisions.

3 Reasons Why Disaster Information is SO Important

For all the talk about about how we need better information delivered in better ways, I am struck by how easily people lose sight of why it is needed in the first place.  After all, information that you don't need is really not helpful and can cause big problems.  You can experience information overload, be distracted from your goals or tasks, or be unduly influenced by extraneous information.  

Whether you are looking for information before, during or after disasters, there are three reasons why good information is important:

1) Situation Awareness - Information helps responders understand the situation

Information helps us identify operational gaps and enables us to effectively coordinate resources.  Knowing the gaps along with what is going on and who is doing what is the heart of situational awareness.  Without good awareness, though, it can feel like you are making a decision in a vacuum or hedging your bets on risky decisions more than you would like.  Generally speaking, the more situational awareness you have, the more you know that you are not duplicating effort and are prioritizing the right issues. 

Good situational awareness strikes the right balance between information deficiency and information overload.  It also incorporates three levels of information (Endsley, 1995):

1. Perception - Knowing the elements of the situation and environment that are pertinent to your job.  They can include things like situation monitoring and cue detection that helps inform your own picture of what is happening.
2. Comprehension - Relating and synthesizing the information from Level 1 against the decisions you need to make and your overall objectives.   At this state, you put your perception information into context with your response objective(s) and decision(s) to better understand the situation.  
3. Projection - Relating to what might happen and one's ability to see the possible trajectories of the events taking place or decisions you make.  You need to have perception and comprehension of the current situation in order for future projections, anticipations and estimates to be effective.  

2) Decision Support - Information helps responders make better decisions 

For some decisions, a general knowledge of what is going on and who is doing what is not sufficient.  General information about the situation won't suffice as you need more specific and focused information to make complex and important decisions.  The goal is to make the best decisions given the information at your disposal that will result in the best outcomes.  For example, choosing to initiate a community-wide evacuation is a big decision with many variables.  A decision support tool that manages information effectively can help you project the impact of a storm (and impact of evacuation) and help you make the decision whether evacuation is the best course of action.  Additionally, knowing the capabilities of and the potential for secondary impacts (i.e., residual flooding) to possible shelter locations will greatly inform your choice of shelter openings.  

3) Outcome Metrics - Information helps us know if we are [being] successful

Information is the key commodity in the feedback loop that shows the failures/success of our decisions and interventions.  Just because a decision is made doesn't automatically mean that the decision and approach is effective.  Continuous feedback is important to take corrective action in a timely manner.   But information doesn't just show success OR failure.  The information that is most useful is usually in between the two in some fuzzy area that requires contextualization and understanding of the dynamics of the situation.  Outcome metrics in disaster management is very difficult and incorporating them take time and experience.  

For example, what if you begin to experience an unanticipated surge of people on the road at 5pm that are clogging your 2 major arteries?  Is evacuation a complete failure?  No.  But does this perhaps require a different approach?  Perhaps you might add additional law enforcement and tow trucks to help keep traffic moving or develop an on-the-spot traffic management plan for a 3rd artery.  But your pre-established metrics indicate that you can still safely evacuate everyone in time because 10,000 people have already left and there is only 3,000 more to go.  As a result, your decision to re-appropriate resources might change.  (NOTE:  This is merely an example.  I am not advocating for letting people "sit" in their cars on the highway if that can be avoided.  This also does not fully address the breadth of "outcome metrics" and how they can be used most effectively)      

4) BONUS - Information as aid  

Collecting information for your own situational awareness and decision making is a rather selfish way of thinking about information.  Information is really a form of aid to the public as well as your response partners.  Information is vital to their success as well.  In addition, disseminating good information in the right formats (i.e., machine readable, real-time access, social media, etc.) for your audience(s) helps improve the community's outcomes as well.  They need information just as much as you do to accomplish their priorities and objectives.  The more they are self-sufficient and armed with the right information, the less burden you carry to take care of others.  

Does this jive?  Why else is information important to you?  

[NEW REPORT] Using Social Media for Enhanced Situational Awareness and Decision Support

After a long effort by the DHS Virtual Social Media Working Group (VSMWG), the report Using Social Media for Enhanced Situational Awareness and Decision Support was just released on Tuesday.  I was very happy to be a part of developing this report among many other talented people.  

“The report introduces ways social media platforms can be used for situational awareness in public safety. It addresses various challenges associated with the use of social media for situational awareness, the integration of social media within the operational environment, and identifies areas requiring further consideration, research, and development.”

The report also discusses issues that extend well beyond social media.  The report highlights many of the issues we face related to disaster information management as a whole.  If you have questions about big data, social data, data interoperability, this report is your primer.

“[The report] also identifies critical areas requiring further consideration and research to address key technology, process, and policy gaps, including:”

• "Information Application: The ability to access, share, search, verify, contextualize, and manage available information. This concept also includes the identification of essential elements of information in social media as they relate to traditional public safety information requirements. 
• Privacy, Legal, and Security Challenges: There are several challenges associated with the use of social media for situational awareness, especially with regards to user privacy and the use of personally identifiable information (PII); the need to remove details when sharing information across multiple partners; and the security of networks, platforms, tools, and data. 
• Data and Open Standards: To truly enhance situational awareness, social media must be integrated, both technically and contextually, within the larger information environment and into the public safety operational workflow. Additional considerations include event detection, data formats, data models, ontologies, semantic and linked data, automation, and artificial intelligence. 
• Technology Development: Challenges associated with the use of third-party platforms, analytics tools, the development of operational requirements, the ability to geo-locate information published to social media, spatial-temporal characteristics (disparate and virtual communities, time decay of posts, etc.), and integration with NextGen911 will require further research. "

The DHS VSMWG also published a great report in May 2013: Lessons Learned: Social Media 
and Hurricane Sandy.  This group is doing great things and trying to push the ball forward with social media and technology.  

I hope you take the time to read through this report.  Look forward to you comments and feedback!

Really, What is Situation Awareness?

We talk about situation awareness a lot. In fact, it permeates a lot of our central assumptions and decisions.  It also impacts how we strategize about our operations and planning, especially as it relates to information management. So really, what is situation awareness?  How does it relate to information management?

Situation awareness is the primary conceptual tool that disaster personnel use to manage all the information that disasters create. Endsley (1988) describes situation awareness as “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future.” According to Endsley (2000), situation awareness is a distinct stage from decision making and the subsequent performance of actions. Endsley (1995, 2000) defines three levels of situation awareness leading to decision making and the performance of actions:

• Level 1 – Perception of current situation,
• Level 2 – Comprehension of current situation, and
• Level 3 – Projection of future status.

Harrald and Jefferson (2007) add that in emergency management, data interoperability leads to a common operating picture, which then leads to situation awareness. These concepts are intricately linked and build off one another. Endsley (2000) points out that situation awareness “is a state of knowledge about a dynamic environment. This is different than the processes used to achieve that knowledge.”

As situation awareness is achieved, though, employing a decision aid such as John Boyd's OODA loop helps decision makers make the right decisions. Originally developed for military application, the OODA loop has application across a wide range of situations.  It has four central components: 1) Observe, 2) Orient, 3) Decide, 4) Act.

So if our goal is to achieve good situation awareness so we can make better decisions, where do we begin?

Researchers such as King (2005) focus on how knowledge management concepts can be used to categorize disaster information needs of decision makers. Knowledge management has close ties to situation awareness as they both deal with the effective use of information. However, situation awareness uses knowledge management concepts to inform decision makers specifically in highly dynamic and mission critical environments. According to the King (2005), “[k]nowledge management is the systematic process and strategy for finding, capturing, organizing, distilling and presenting data, information, and knowledge for a specific purpose and to serve a specific organization or community.”  He bases his categorization on the needs of decision makers as a whole and finds there are four different types of knowledge types that decision makers seek out:

1. Situational
2. Background
3. Operational
4. Analytical

This categorization is quite high level, though, and does not fully address the range of decision making information needs. In addition, the ability to find, capture, organize, distill and present data, information and knowledge is quite challenging in practice. King (2005) points out the challenges organizations face:

“Information is constantly changing, comes from a multitude of sources and is often incomplete or contradictory. In some cases, there is an overload of information and, in other cases, there are complete gaps in what we know. Collecting information is often difficult, if not impossible, because of inaccessibility to the affected areas due to natural hazards, insecurity or government restrictions.”

Zhang et al. (2002) adds:

“Relief agencies are slowly developing the infrastructure to undertake effective information and knowledge management. A huge amount of information is collected but not efficiently used. Despite advanced technology achievements, many decision are still taken in emergencies with little information beyond that in people’s heads.”

Information management is inherently complex. Overcoming this complexity is a key challenge for our future as more and more information becomes available, especially in real-time.  So what approach will yield the greatest effort-to-outcome ratio in improving information management?  I think the we still have a ways to go on this, but simply categorizing and typing all available information is also impractical, costly, and causes strategists and technologists to lose focus on important and high value contributions.

So what are your thoughts on situation awareness? Does this jive with your interpretation? What are the challenges you face in practice?