Issues and Challenges Pertaining to Large Scale Data Collection

1

• Issues and Challenges Pertaining to Large Scale
  Data Collection
• By Jeremy LaDart
• US Army Corps of Engineers Mobile District

2
Mississippi Coastal Improvements Program

• 10 Million Emergency Supplemental Appropriations
• (P.L. 109-359) 30 December 2005
• Cost Effective Projects in lieu of
• NED benefits
• No Incremental Benefit-Cost Analysis
• 6 month interim and 24 month comprehensive report
  requirements
• Hurricane and Storm Damage Reduction
• Salt Water Intrusion
• Shoreline Erosion
• Fish and Wildlife Preservation
• Other Water Related Resource Projects

Studies related to the consequences of the 2005
hurricanes All efforts fully coordinated with
the Louisiana Coastal Protection and Restoration
Project (LACPR) team
3
Six Step Planning Process
4
Steps for Large Scale Data Collection

• Determine Extent of Study Area
• Define Data Needs
• Determine if Data Currently Exists
• Determine Validity of Existing Data
• Identify the Collection Method and Acquire Data
• Compile Data for Use
• Create Metadata

5
Major Issues

• Monetary and Resource Limitation
• Time Constraints
• Scale/Scope Can be Overwhelming

6
Major Challenges

• Safety of Team Members
• Cooperation of Agencies
• Availability of Existing Data
• Meets Desired Confidence Level
• Data May Not Overlap Your Area

7
Determine Study Area Extent

• Maximum Area of Evaluation
• Will Dictate Your Data Collection Methodology
• Can be Subdivided into Multiple Levels
• State, County, City, Census Tract, Census Block

8
Examples of Economic Data Needs

• Immediate Recovery Statistics
• Socio-Economic Characteristics
• Population
• Income
• Employment/Unemployment Rate
• Structure Characteristics
• Location
• Occupancy Type
• Extent of Damage
• Value (Structure and Content)
• First Floor Elevation

9
Existing Data Sources

• Sources of Existing Data
• Local, State, and Federal Government Agencies
• Academia
• Non-governmental Organizations (NGOs)
• Issues and Challenges
• May not Perfectly Overlap Your Study Area
• Cooperation of Agencies
• Verify Quality and Validity of Data
• Meets Desired Confidence Level

10
Data CollectionDetailed Field Analysis

• Typically a Feasibility (Census) Level Analysis
• Team Members Carefully Drive the Area and Collect
  Data at the Structure Level
• Analyze Entire Population of Data
• Data Should be Extremely Accurate
• 95 percent confidence level

11
Data CollectionDetailed Field Analysis (Cont)

• Pros
• More Accurate than Sampling
• Often the Preferred Method of Data Collection
• Issues and Challenges
• Extremely Time Consuming
• Extensive Money and Resources
• Safety of Team Members

12
Data CollectionSampling Techniques

• Random Sampling Techniques
• Simple Random
• Stratified Random
• Cluster
• Multistage
• Systematic

13
Data CollectionSampling Techniques

• Pros
• Faster than Detailed Field Analysis
• Requires Less Money and Resources
• Issues and Challenges
• Not as Accurate as Detailed Analysis
• Utility and Confidence Level is Limited to
  Existing Data
• Increased Accuracy Increased Complexity

14
Compiling Data for Use

• Creation of a Database is CRUCIAL
• Microsoft Excel and Access
• Linux
• GIS
• Web Based
• Issues and Challenges
• Quality Control is a MUST

15
Create Metadata

• Metadata is Data about Data
• Important for You and Other Users
• Examples of Metadata Include
• What is the Data? (structures, content, etc.)
• Where Did the Data Come From?
• What Was the Data intended to be Used For?
• When Was the Data Collected?
• Who Can Have the Data?

16
MsCIP ExampleExtent of Study Area
17
MsCIP ExampleTax Parcels in the MPI
18
MsCIP ExampleMagnitude of Scope
19
MsCIP ExampleMagnitude of Scope

• 1,361 sqmi Area (100 sqmi Larger than Rhode
  Island)
• Over 200,000 Parcels
• 800 11X17 Parcel Maps
• Over 3,000 man-hours

20
Data Collection MsCIP Example

• Cluster Sampling Technique with Vigorous Field
  Analysis
• Team Members Drove Every Street within the MPI
  Area (1,361 sqmi)
• Areas were grouped by Blocks, Neighborhoods, etc.
• Field Collection was Conducted by Group

21
Data Collection MsCIP Example

• Pros
• More accurate than Sampling Alone
• Faster than a Detailed Field Analysis
• Issues and Challenges
• Not as Accurate as a Detailed Field Analysis
• Safety of Team Members
• All Team Members MUST be on the Same Page

22
MsCIP Lessons Learned

• Get Everyone on the Team Involved Early
• You can NEVER over Verify Existing Data
• Have a Defined QC Plan Up Front
• Field Journal for Every Team Member

23

• QUESTIONS?