Windows and UNIX Interoperability tips, tricks, and secrets

1
Windows and UNIX Interoperability - tips,
tricks, and secrets

• Peter Skjøtt Larsen
• Lead PM
• Microsoft Corporation

2
Vision In 5 years, most enterpriseswill have
Windows and .NET central to their platform
strategyfor business computing
3
More than an operating system
4
Computing Reality

• There is a natural separation between
• Server side rigorous expensive high performance
  computing
• Client side human-time intensive flexible
  modeling, planning, programming, etc.
• Most UNIX based scientific applications require a
  UNIX environment for client side activities.
• Code is written for X or use special libraries,
  scripting languages, UNIX text processing.

5
Windows Attraction

• Client Side
• Many sophisticated and volume priced data
  management apps available on Windows
• Person-time consuming aspects of scientific
  computing is conveniently done on laptops.
• Server Side
• Windows is being used increasingly for diverse
  server side workloads.
• Windows is now a highly scalable serious workload
  contender (great TPC results)

6
Client Options for UNIX code

• A number of alternatives exist today
• Improved UNIX clients with better applications
• Better desktops apps for Linux, etc.
• UNIX like environments on Win32 API
• Cygwin, uwin, mks
• UNIX emulation on Windows Kernel
• Microsoft Services for Unix
• Virtual Machines
• Microsoft Virtual Server
• Windows like environment on UNIX
• Wine

7
What Is Services For Unix?

• Interoperability
• NFS Client, Server, Gateway
• Password Synchronization
• NIS/Active Directory migration
• Portability for Unix
• Scripts
• Applications
• Users
• Supportability
• Commercial MS product

8
All the comforts of home

• Replaces Posix subsystem (in Windows)
• C Shell and Korn shell
• Single-rooted file system
• Symbolic links
• Win32 programs
• Terminals and other devices
• Services and daemons
• Man pages
• X windows

9
Windows And SFU
UNIX Applications
X11 R6 server
Windows Appli- cations
Windows Appli- cations
Motif
UN I X S D K (gcc)
Open Source tools Apache, Tcl/Tk, bash, etc.
Windows GUI
X11
UNIX, XPG, POSIX.2 commands utilities
UNIX shells
Windows system admin, commands networking
Windows command Shell
SFU/Interix
telnetd
3rd Party
BSD Sockets
winsock
Windows APIs
UNIX /POSIX APIs
Color Legend
10
Infrastructure NFS
11
Mapping Windows to UNIX
Request Windows to UNIX mapping
NFS access granted
Return UNIX credentials
NFS request granted using UNIX credentials
NFS request sent using UNIX credentials
12
Mapping UNIX to Windows
User john_smith in Unix domain
NFS request
Unix Machine II
NFS access
NFS request sent using Windows credentials
Mapped Windows users access provided
User Name Mappings
Authenticate mapped Windows user
Windows domain
13
InfrastructurePassword Synchronization

• Bidirectional - Change passwords from Windows NT
  or UNIX
• Fine grained control - updates filtered by user
  name
• Password updates propagate to selected systems
• Passwords Triple-DES encrypted in transit
• Limited to users with identical names

14
Infrastructure Printing

• Basic steps
• On Unix system, define and configure aremote
  printer
• On Windows, install Print Services for Unix
• Unix lp and lpr commands can be used
• Standard Unix printing protocols do nouser
  authentication
• Secure by client IP address

15
BSD /etc/printcap

• lp1\
• of/usr/lbin/pcfof Chp5mp.pcf\
  sd/usr/spool/lpd-hp5mp\ lp\
  rphp-5mp\ rmwinsvr1
• of filter driver (omit if unnecessary)
• sd spool directory
• lp printer device name (empty for network
  printers)
• rp remote printer name (windows share name)
• rm remote system name or IP address

16
Set up Windows printer

• Install and create a share in the usual way (e.g.
  \\winsrv1\hp-5mp)
• Install Windows Print Services for UNIX
• Add/remove Windows Components Other Network
  File and Print Services Print Services For UNIX
• Enables server support for BSD/LPR printing
• Also enables printing from Windows to remote UNIX
  printers via lpr
• Check Services Manager to ensure TCP/IP Print
  Server (tcpsvcs) is running

17
SFU for the developer

• Command line environment
• Korn/Bourne shells, vi, more, etc.
• awk, grep, perl, sed, etc.
• at/cron/batch, rlogin, telnet, etc.
• Application SDK and runtime support
• POSIX.1, POSIX.2 (IEEE 1003, ISO 9945)
• Most of UNIX98
• gcc, rcs, make, etc.
• cc/c89 tied to MSVC

18
Whats New in SFU 3.5

• Pthreads support
• Threading
• New synchronization primitives
• Internationalization support
• I18n APIs
• Limited localizability for some utilities
• X11R6.6
• gcc 3.3
• Performance improvements

19
Cool Tricks with Threads

• Although Interix apps cant call Win32 APIs, they
  can call ntdll.dll
• WARNING! Microsoft doesnt support direct use of
  the NT kernel interface
• However, there are several books on Undocumented
  Windows NT Calls
• If you create a separate thread and mask all Unix
  signals in that thread, making ntdll calls is
  relatively safe
• Test the heck out of this!

20
Now, Why Would You Do That?

• I/O to devices not exposed by Interix
• Use NtOpenFile, NtReadFile, NtWriteFile,
  NtDeviceIOControl, etc. to talk to the device
• Using NT synchronization primitives
• Mutexes, critical sections, events

21
Flow of Control
ux_thr
nt_thr
Block all signals except SIGUSR1 sigpause() Bloc
k signals pick up request Make desired calls (do
I/O, wait on synchronization object,
etc.) pthread_kill(ux_thr, SIGUSR2)
Set up request data in known mem pthread_kill(nt
_thr, SIGUSR1) Block or do other work until
SIGUSR2 is delivered Signal handler for
SIGUSR2 invoked
22
Building Against ntdll.dll

• Cant use Interix build tools
• Compilation trips over headers
• Cant link Interix objects directly against ntdll
• Solution Build your own DLL
• Contains just functions which call ntdll.dll
• Be sure you dont touch kernel32.dll
• Use VS to build and link generate FOO.LIB file
• Interix code links against FOO.LIB file
• Rename to foo.a to make ld happy

23
But, I want to call Win32 !

• In a coexistence scenario it is possible that an
  application on the SFU side can benefit from
  calling functionality on the Windows side (and
  visa versa).
• Such cases must be carefully studied for the
  performance impact of marshalling and the extra
  development cost involved.
• Low volume communication between loosely coupled
  functionalities are good candidates.

24
Creating a Proxy DLL

• All DLL Functions are duplicated with stub code
  on the SFU side.
• All calls go through a marshalling mechanism to a
  DLL host on the Windows which loads the DLL and
  calls the function.

Could be RPC
SFU APP
Win DLL Host
SFU Std. Libs
Proxy Lib
Win DLL
Marshalling Lib
Marshalling Lib
Services for UNIX
Windows
25
Almost shared memory

• A tighter coupling can be achieved with memory
  mapped files.

SFU APP
Win App
SFU Std. Libs
Win Std. Libs
Services for UNIX
Windows
NTFS
26
Call Win32 App from SFU, Easy!

• exec() will naturally invoke Win32 applications
  and map stdin, stdout and stderr as expected.
• The shell will also seamlessly start Win32
  applications.

27
Call SFU App from Windows

• Windows has access to SFU commands by invoking
  the SFU command directly.
• The Posix.exe command is designed to translate
  between path formats and invoke the SFU
  subsystem.

28
Porting Satisfying Dependencies

• configure around missing libraries
• Search for latest versions of required libraries
• http//google.com
• http//freshmeat.net
• http//interopsystems.com/tools
• Expect dependencies to have their own
  dependencies in turn

29
Correcting Compile Errors

• Use ifdef _INTERIX / endif
• Look for existing portability ifdefs
• Most are really portability bugs
• Look for similar functions in POSIX, SUS
• Consider platform assumptions that arent
  guaranteed to be true everywhere
• Disable sections of code to get something running

30
Common Problems

• Missing functions or constants
• Check for _ALL_SOURCE
• Search for standard equivalent
• Missing ioctl constants
• Look for equivalents
• Missing functions at link time
• Check order of libraries in link command
• Functions may appear in different libraries
• Some core functions appear in libpsxdll.a instead
  of libc.a

31
Running Unix/Linux Applications in a Virtual
Machine

• When is Virtualization Appropriate?
• No logical way to run app on Windows or SFU
• App and supporting libs exist on Linux

32
Managed Co-Existencewith Virtual Server
Windows APP
NT 4.0 APP
UNIX APP
Virtual Server
Windows 2003 API
NT 4.0 API
UNIX API
Windows 2003 Kernel
NT 4.0 Kernel
UNIX Kernel
Virtual Server
Hardware Abstraction Layer
33
Virtualization Results

• Linux app runs in the Windows environment with
  integrated
• User file store
• Security context
• Command execution environment
• Access Linux transparently from Windows
• Linux / UNIX apps run out of the box
• Performance acceptable for many classes of apps

34
UNIX Application Migration Guide

• ANSI C/C and Fortran Applications
• Architecture Differences of Unix and Windows
• Code Assessment Guidelines
• Guidelines and Best Practices on
• Prototyping
• Implementing
• Testing Code
• Code samples illustrate key concepts
• Freely available on MSDN
• http//msdn.microsoft.com/library/en-us/dnucmg/htm
  l/ucmglp.asp

Exposing The Best Known Practices From Field
Experts
35
Unix Migration Guides
36
All Technology Options
Making the move from UNIX to Windows easy

• Visual Studio .NET
• Simplify evolving UNIX code to Windows .NET
• Code conversion tools
• UNIX tools libraries
• Make file conversion

• Services for UNIX
• Complete UNIX compatibility
• Full real UNIX scripting
• APIs/tools for Solaris / Linux compat
• Tight integration with Windows

Guidance Migration guides Automation
tools Scenario focused Community Community forum
support Customer contributions Shared tools

• Virtual PC Virtual Server
• Run UNIX binary apps
• Minimal support for selected scenarios
• Selected guidance
• Respond to customer usage needs

• Interop with UNIX
• Seamless integration with UNIX legacy
• Better interop during legacy transition
• Support UNIX clients for Windows, SQL
• Leverage BizTalk, SFU, .NET interop

37

• More info
• http//www.microsoft.com/windows2000/migrate/unix
• Email
• migrate_at_microsoft.com
• petela_at_microsoft.com