FORTRAN Short Course Week 4

1
FORTRAN Short CourseWeek 4

• Kate Thayer-Calder
• March 10, 2009

2
Topics for this week

• Searching in Unix
• Grep, Regular Expressions
• Multi-Dimensional Arrays
• User Defined Datatypes
• Missing data
• Reading and writing scientific data

3
Unix Wildcards

• - matches all files with none or more of the
  pattern
• ls a returns all files ending in a
• ls a returns all files starting with a
• ? - matches exactly one character
• ?ouse would return house and mouse, but not
  grouse.

4
grep

• Searches through a file looking for a specific
  string or pattern, returns the lines where the
  string occurs
• grep -i alien ufo.txt (case insensitive)
• grep -w abduct ufo.txt (whole word only)
• grep -riw saucer (recursively thru
  subdirectories)
• or lines where it does not occur
• grep -v censor ufo.txt
• Can grep multiple files, just add them to the
  list on the line grep -i parameterization
  .txt
• Unix lexicon Cant grep dead trees.

5
Regular Expressions

• aka RegEx, is a special string for describing a
  pattern of text
• RegExs can be used with grep or other unix
  commands and programs for sifting through text
• They can get really huge, confusing, and
  powerful, well just look at a few simple
  options.
• For more http//www.regular-expressions.info
  or just man regex

6
Grep and Regex
7
grep combinations

• Just some ideas for using grep with other Unix
  commands
• ls -al grep Jan
• ps -ef grep 501
• man ftp grep -i directory
• head -30 mydata.txt grep temperature

8
But...

• Searching has become much easier than it once
  was, usually your desktop search engine will
  filter through files looking for your keywords
• So, lets talk about more Fortan!

9
Multi Dimensional Arrays

• type, dimension(dim1,dim2,...) name
• REAL, dimension(lon,lat,height,time) temp
• Higher dimensional arrays are usually stored
  contiguously in memory or binary files, in COLUMN
  MAJOR order
• See example Multiarrays.f90

10
Column Major

• Fortran fills up each dimension in order
• So for i,j,k array, i fills first, then j, then k
• But do loops work inside out
• Write out k first, then j, then i
• To fix this, write your do-loops from the last
  index in to the first.
• Do time1,days
• Do lon1,360
• Do lat1,180
• Read (10,fmt) Data(lat,lon,time)
• enddo
• enddo
• enddo

11
Array Transformation
lons

• Reshape function is pretty cool
• Matrix RESHAPE( Source, Shape )
• A RESHAPE( B, (/3,2/) )
• Another way to index your array elements uses
  mod and integer division
• lat array(MOD(i,num_lats)1)
• lon array(i/num_lats 1)

lats
12
Allocatable Arrays

• Sometimes, you dont know how large you want your
  array to be until runtime.
• Fortran 90 has allocatable arrays that can be
  declared without fixed dimensions, and filled in
  when the program is running.
• These can be filled from stdin, or a variable in
  a file, or a calculation based on previous work,
  or any other run-time value.
• See example Multiarrays2.f90

13
WHERE statements

• An easy way to initialize or set sections of
  arrays
• WHERE (array expression)
• array assignment block
• ELSEWHERE
• array assignment block 2
• END WHERE
• This is called masking

14
FORALL Construct

• This statement indicates to the compiler that the
  operations can be performed in parallel (no
  operations depend on the value of the operation
  on other elements in the array)
• FORALL (triplet)
• variable expression

15
Atmospheric Data

• Youll see data stored in arrays in many ways
• MyData(pressure, temp, mixingratio, height)
• MyPressure(height), MyTemp(height),
  MyMixingRatio(height)
• Pressure(lat,lon,height,time), Temperature(lat,lon
  ,height,time)

16
The Perils of Parallel Arrays

• It is common in our science to see people using
  multiple arrays of data that are all the same
  shape but for different variables (Temperature,
  Pressure, u wind, v wind, ...)
• This is considered bad form in computer science,
  it would be better to have one array with
  multiple values possible at each point. Why?
• This gets confusing if you implement a 5-D array,
  however.

17
User Defined Data Types

• Fortran gives us a nice way to describe more
  complex data structures by creating new data
  types.
• Instead of 4 arrays with different variables in
  each, we can have one array with four values at
  each point.
• TYPE name
• DataType Component_name
• ....
• END TYPE name
• We can create variables with this type or arrays
  of variables of this type
• TYPE (name) VariableName
• TYPE (name), Dimension(d1,d2,d3,d4) ArrayName
• Example StdAtmos.f90

18
IEEE (Eye-Triple-Eee)

• Institute of Electrical and Electronics Engineers
• Most members of any technical professional
  organization in the world.
• IEEE produces standards for almost everything
  computer related, and keeps them updated.
• Today, most IEEE standards have generally been
  adopted.
• IEEE 754 is the most widely-used standard for
  floating-point computation, and is followed by
  many hardware (CPU and FPU) and software
  implementations

19
INF and NaN

• INF is defined as the value given to any Real
  that is outside the limits of the type.
• Fortran has INF and -INF
• NaN (Not a Number) is produced as the result of
  an improper floating point calculation.
• NaN is not equal to either INF. In fact, in the
  IEEE standard, NaN is not even equal to itself.
• INF or NaN are occasionally used as placeholders
  for missing data.
• See Example WriteExample2.f90

20
Missing Data

• Any observational dataset is going to have holes.
  
• If missing data is not given as an outside the
  bounds value (-9999 or 9999.0) it is often
  replaced with INF or NaN.
• Most Fortran implementations will read INF or NaN
  in as a Real value (it is a real Real), we need
  to check for it before doing calculations, or
  well get a runtime error.
• See Example ReadBadData.f90

21
NetCDF Data

• NetCDF is an I/O library that is widely used in
  the earth sciences.
• Once the files are installed, you can use their
  procedures to open and access the files
• Each files is self-describing, all of the data
  is annotated (dimension, units, range of values,
  missing data values, etc...)
• Examples read_netCDF.f90 with data from NCEP
  (NCEP.Precip.0100-1204.nc)

22
Zonal Average Example

• Modelers and Dynamicists like to look at the
  atmosphere in latitudinal bands.
• Dont have to worry about missing data here...
• Loading in precip data is pretty simple if you
  know the parameters.
• When you do a zonal average, first average in
  time at each point and then average across all
  longitudes.
• Could come up with a less memory intensive way to
  get the same result...
• Example PlayWithPrecip.f90

23
What did we talk about?

• Searching in Unix
• Grep, Regular Expressions
• Multi-Dimensional Arrays
• User Defined Datatypes
• Missing data
• Reading and writing scientific data