CDROM.txt 6.8c    HOW TO BUILD APRS MAPS FROM THE USGS CD ROM    5 May 95

New in 6.8c: New MAKEMAPS.bas can make a 2,000,000:1 map almost hands free!

New in 6.8b: I found a smoother way to process the 2,000,000 maps. See the
             32 mile map in North Tennessee.  See section at end of this file
             New improved CONV100C.bas does NOT crash on city streets!
New in 6.7:  Brought README.100 into this file to reduce the # of txt files


NOTE:  Also look for special guidance for mapmakers in README\MOREMAPS.txt

WARNING:  Trying to make an APRS map from the 100,000:1 USGS CD ROMS should
be considered EXPERIMENTAL and quite TIME CONSUMING at this point.  Do not
consider doing it, unless you accept a challenge or think you can help
us find a quicker way to do it.  For now, it is more fun to me to use
a DIGITIZER (if you have a paper map) and take a full evening, than the
time involved in the multiple passes with the tools that follow... If
however, your digitizer is only 11 inches square and you have to re-place
the MAP 4 or more times to get it all in, then I would say it is a toss-up.

BEGIN:  All information on using the original 2,000,000:1 USGS CD ROM has
been moved to the end of this file, since by now, (fall 94) almost all
of the US population centers have at least the ROADS from this data base.
NOW the focus is on the use of 100,000:1 USGS CD ROMS
that contain data on roads all the way down to your street.  Because of the
tremendous amount of data available in this data base, USGS sells the data on
a set of 13 CD's.  To see which CD covers your area, hit 1 on your APRS screen
to get to the whole USA map.  Then press MAPS-ALT command to see the CDROM.map
file which is under the USA map.  This display shows you the areas of the 13
different ROMS.  My yellow lines are crude, but the data in the files does
truncate exactly at the state lines.  To obtain one of these CD ROMS, send $32
for each ROM to the USGS Earth Science Info Center, 507 National Center,
Reston, VA 22092. 703-648-6045 or call 1-800-USA-MAPS.
BE SURE TO SPECIFY THE OPTIONAL FORMAT!!!!!!!!!!!!!!!! (vice standard)


JUST BECAUSE WE HAVE NOW WRITTEN ROUTINES TO HELP YOU MAKE MAPS FROM BOTH THE
2,000,000 and 100,000 SCALE USGS CD ROMs, DO NOT GIVE UP ON MAKING YOUR OWN
MAPS USING ANY OF THE OTHER METHODS.  THE 2,000,000 SCALE CD IS ONLY GOOD FOR
64 MILE MAPS because THEY DONT HAVE ENOUGH DETAIL TO MAKE SMALLER MAPS, and
they HAVE TOO MANY POINTS TO MAKE MAPS MUCH LARGER!  The 100,000 scale maps
generate so many points that an APRS map of city streets can be no larger than
about 1 or 2 miles in dense neighborhoods!

BACKGROUND:  There are 3 problems with the USGS data that causes excessive
points in an APRS map.  First, lines begin and end at EACH intersection.  For
am interstate, for example, you end up with dozens of short road segments
wasting 3 point pairs per exit.  Second, these short segments are not always
consecutive in the file, making it extremely time consuming to try to join
them.  Third, the USGS data guarantees 100 points to the inch at the original
maps scale (1/3 mile for the 64 mile maps) whether it is needed or not.  A 10
mile long straight road is stored as 30 points in USGS vice 2 points in APRS.
At the 100,000:1 scale, .01 inch is only 100 feet!


VERY FINE DETAIL USING THE USGS 100,000:1 DETAIL CD ROMS

    W7KKE wrote a program called CONV100c.bas which takes data from the
100,000 scale CD ROMS.   It generates so many
roads in an urban environment, that just a 1 square mile map has over 5000
points.  I have upped the APRS maximum limits to accomodate up to 2999 points
but you should avoid making these large size maps unless you absolutely
need every street, because the large size also means APRS takes longer to
load maps!  Since we are vehicle tracking for radio purposes, a whole map
1 mile square is only one one-thousandth of the potential coverage area of
one APRS digipeater, so you can see that the effort to make these maps is
not worth the time in my opinion. (unless you are making the detail for
a commercial application and are willing to spend the time!)  Here is how
to do it:

1.  Run the USGS EXTRACT program and follow its instructions to output the
appropriate map data for the 15 x 15 minute area that you are interested in.
These files will be of the form XXXRDSnn for ROADS and xxxHYFnn for hydro
where the nn is for multiple files to cover the area.  In my urban area,
each of the ROAD files covers about a 2 mile area.

2.  Run CONV100c.bas and once you give it one of the above file names, it will
automatically generate a series of 10 output files of the form PREnRD.GRF for
n different ROAD files and two other files preWB.grf and preST.grf for
waterbodies and streams.  100toGRF.bas outputs these files in the same format
as the original 2,000,000 that MAPFIX knows how to read.  The PRE is a 3 or 4
letter prefix that you specify for this series to keep it separate from other
efforts.  So far, the number 4, 9 and 10 maps in my area have been empty...
but the pre8RD.GRF file is the largest by a factor of ten!  It contains all
of the neighborhood streets.

3.  Run MAPFIX and load an existing APRS map that covers the area where
you will be working.  This map just makes it easy to visually get the
cursor to the right area.  Once you hit the ALT-U commmand, this existing
MAP is no longer used.  MOve the cursor to the center of the map you want
to make, and hit the B command to establish the boundary of the area
that you want to develop.  Choose a scale of 2400, 3000, 3600, 4800 or so
that will cover the total area you eventually want covered.  For city
streets, you should use at least 2400 or more.  Next use the alt-USGS
command to bring in the first of the above files.  Then do the alt-SMOOTH
and the alt-JOIN commands to minimize the map.  Then SAVE IT WITH A NEW
NAME.  Then repeat this step for all the maps that were output from the
CONV100.bas program.  After you have saved each of these, you will now
have almost a dozen maps each containing a different feature, but in APRS
format.

WARNING!:  NEVER do a second ALT-U USGS BUILD on the same map, because it
will clear all current points and start a new map from scratch.

4.  Next, run Ken's DEFRAG.bas program on EACH of the files generated by
MAPFIX in the above step 3.  This program goes through the files over and
over again, trying to join together all of the disjointed road segments
(of the same class) generated by the USGS data base.  Hopefully you may
see a 20% or more reduction in points.  Re-save each of these reduced
maps under a new name so that you do not distroy your originals (if there
is a problem.

NOTE:  Be sure to also review the steps at the end of this file on how MAPFIX
is used to read the 2,000,000:1 CD ROMS.  Since the 2,000,000:1 process was
developed first and is the basis for MAPFIX routines, you might find some
helpful details or clarifying information.  Ken Swaggart, W7KKE, wrote the
CONV100c.bas program to put the 100,000:1 OPTIONAL data format into the
GRAPHICS format which is then readable my MAPFIX.bas.

His instructions are in README.100, NOW INCLUDED AT THE END OF THIS FILE.

4.  Now, combine these maps into one APRS map, using the MAPFIX alt-IMPORT
command.  This command will bring just one, or all of the map features of one
file into the current file.  In an urban area, the level 8 road file will be
larger than all the others combined.  You can omit it to allow your map to
cover a larger area or include it for the finest detail available.

Good luck.  I am just beginnning to play with this level of detail, and to
this point, it is quite practical to use a digitizer!  For more detail on
the use of MAPFIX with CD ROMs, also read the 2,000,000:1 section below,
since it evolved and was the basis for APRS maps before we began messing with
the 100,000:1 data.



PROCEDURES FOR 2,000,000:1 CD ROM THAT COVERS THE WHOLE US

Most of the initial 64 mile and larger maps were made from this CD.  But
to make them small enough, all County lines, most streams, most lakes and
railroads were omitted.  Also, many points were eliminated in the roads.

*** Conversly, we NOW (May 95) have some new tools available to make these
2,000,000 type maps (including county lines and streams) relatively easily.
The first such new map that I made that includes EVERYTHING that is on
the 2,000,000 CD is in the North Tennessee map, TNnorth.map.  Note that it
is very full and only covers about 32 miles square. 

The new precedure uses the MAKEMAPS program written by KB4XF that XTRACTS
data directly from the CD and saves it in APRS format.  You simply choose
the state and specify a LAT/LONG and range for the center of the map.  By
choosing a small enough area (40 miles or so in the east, maybe 70 or more
in the rural west) you get a resulting map in the 4 to 5000 range.  THen
load this into MAPFIX and use the alt-SMOOTH command successively to get
the number of points below 3000, AND YOU ARE DONE!!!

The following STEPS outline the older method which is pretty obsolete. BUT
you may still use the following techniques to get SINGLE features from the
CDROM such as RAILROADS or ADMINISTRATIVE boundareis that are NOT included
in the new MAKEMAPS program.

1.  Use USGS XTRACT.EXE program to extract the CD ROM data in the GRAPHIC 
format for your area from one of the dozen or so major USGS areas.  Then
select up to 6 categories for extraction onto your HARD disk, one for each
of the data types, Political BOUNDARIES, ROADS, WATER BODIES, STREAMS,
RAILROADS, and HYPSOGRAPHY (continental divide).  These files will take
about 5 to 10 MBytes.  Note the filename assigned by XTRACT.

2.  Run MAPFIX.BAS program and load the best existing APRS map that covers
the desired area.  This map is only used to help you locate the area of
interest.  Next, move the cursor to the center of interest and select
a SCALE using the BOUNDARY (B) command.  This command will show you the
maximum map possible for a given scale in Points/per degree.  I have found
600 to be about right.  450 in sparse states and 300 in UTAH and NEW Mex.
If you use any less, to get larger maps, you will end up with far too many
points.  I used 900 for a 35 mile map around Philadelphia.  Making maps any
smaller than this is probably not worth the effort, since there will be
so few roads in it..

3.  When you have the B command set to the desired number of Pixels-per-degree,
press the alt-U key.  MAPFIX will allow you to round the ORIGIN values, to
reasonable nearly whole numbers or fractions of a degree so that later
manual operations with a map and ruler will have round numbers to deal
with.

4.  You do one type of data at a time.
APRS asks you for the type of file (RD, ST, WB, etc) and then the
exact path to the XTRACTed USGS data.  This path should include the first
part ot the filename up to the (_) underline that separates the RD or WB
etc final two characters.  On my system it is usually D:\NEAST\S01_  and
then MAPFIX adds the RD.GRF on the end.  MAPFIX will then scan the entire
indicated file for all points within the selected border.  While converting
the points to APRS format, MAPFIX will also compare the slope of each new
point with the previous point and ignore the point if it has approximately
the same slope (within 1.2)  The total number of points will be from 2000
to 4000.

5.  Next use the alt-JOIN command.  This will cause MAPFIX to scan through
the map looking for consecutive map features that end and then start
at the same point.  These two features will be joined into one longer
continuous feature.

6.  Next use the MAPFIX SMOOTHER command which will run another slope
comparison.  You can select the SMOOTHER factor.  A factor of 1 will only
throw out points that are exactly the same slope (few if any).  A factor of
1.2 (best) will throw out points whose slope differs by less than 1.2.
Bigger values will throw out more points.  If you start with 1.2, next
try 1.3, then 1.4, etc.  Anything above 2.0 or even 1.5 and you can begin 
to see some detail being lost.

7.  At this point, you can save the map using Ctrl-S giving it a name which
includes the feature type (..RD, ..ST, ..WB, etc)  As long as you have not
hit the SPACE bar or changed scales, the original MAP which was used for
oreintation will still be on the screen with all the new dots.  This map is
NOT involved at all, but is still on the screen.  Be sure you have saved
the new map points and then hit alt-U again to do the process again for
another feature type (RD, WB, ST, etc).  If you have not changed anything
your ORIGIN values will be reused again.

8.  Repeat steps 3 through 7 for all 6 data types, being sure to SAVE each
type separately, but without re-drawing the map.  When you are done, quit
MAPFIX and run DEFRAG.bas on the RD file. (there is not much to join in
the others, due to minimal intersections).  This may remove some
additional points by further joining lines.  DEFRAG does not do LABELS,
so if you run DEFRAG on a map with labels, they will be lost.

9.  Now you have the smallest APRS maps that can be done automatically.
Reload MAPFIX, reload the RD file and Move the MAP POINTER to the beginning
FEATURE using the RESET command and use the gray +/- keys to cycle through
EVERY point in the file.  Use the alt-D key to delete all unnecessary points.
Look for complete duplicates of some lines too.  Both lanes of a 4-lane or
interstate are in the file and will be EXACTLY the same!  This should get
you down to below 2000 or so.

10.  Now simply use the Alt-IMPORT command to bring in each of the other
feature maps.  SImilarly go through and remove unnecessary points.  Be sure
the WHITE box is way out beyond the area of the maps, so there is no
TRUNCATION going on with the IMPORT command (It only immports points within
the white box.).  There are usually LOTS of points that you can remove
from the PB, AB, WB and ST files, with little loss of detail.

ALTERNATE PROCESS FOR WB, ST, BP, AB FILES:
(This is the way I used to do it, but NOW I prefer the above method better)
 
    Another way to add some detail from the other non-road USGS files, is to
use the MAPFIX USGS OVERLAY feature to temporarily overlay the USGS data on
the MAPFIX screen without adding points.  Then you use the NEW-FEATURE and
ADD commands to draw APRS features over the top of these temporary points.
The philosopy here is that 96% of amateur interest is on roads, and you
can sure burn up points on waterways and county lines.  For rivers, about
the only points that are important are where the water goes under a bridge.

    You will first need to overlay on the 64 mile scale to see where things
are, but then zoom into the 16 mile scale at least and do the overlay again
to get enough resolution for drawing a particular feature.  Usually, I
sketch in rivers at the 32 mile range, because I do NOT try to get every
kink in a river or stream...  You can use the NEW, ADD and MOVE commands,
but any other commands will redraw the map and wipe-out the overlay.  Just
hit the U command again to reproduce the same overlay.  DO NOT CONFUSE THIS
with alt-U which will WIPE-OUT your map.  Use the Ctrl-R command to toggle
the map REDRAW process on and off. This may help you to add lines without
destroying the USGS overlay.  You can do the overlay again and again.

WARNING!:  NEVER do a second ALT-U USGS BUILD without FIRST saving what
you have, because it will clear memory and start an all new file.


CAUTION:  Do not expect too much from this CD ROM.  ALthough the 2,000,000:1
USGS database seems to have resolution down to 1/3 rd of a mile, remember
that that 1/3rd of a mile was actually 1/100th of an inch on the original
map!  Your 64 mile section was only a two inch square!!!!!  AND THE DATA
WAS PROBABLY DERIVED BY A HIGH SCHOOL SUMMER STUDENT using a mouse!

      I have found this data to be very crude, and although the maps lines
look smooth, you are getting about the same quality as if you were
looking at a state highway map through a magnifying glass!  If you want
good detail, use MAPFIX and a digitizer on a much better original map,
such as a 100,000:1 or even 24,000:1 (which is the standard USGS 7.5
minute map).  There you can get down to your driveway!

IT STILL TAKES AN EXCESSIVE AMOUNT OF MANUAL EFFORT TO REDUCE THESE MBYTES
OF USGS DATA DOWN TO A reasonably quick loading 15K APRS MAP!  IT TAKES ME 4
to 12 HOURS PER 64 mile MAP!  About 3 hours for 32 mile maps..  By now
(Fall 94) I have done most of the US to the state highway level of detail.
IF YOU WANT ME TO DO A SPECIAL MAP OF YOUR AREA, to the resolution in the
2,000,000:1 ROM, SEND $$$ or convince me it is in the general best interest
of all HAMS.  THAT ALONE SHOULD KEEP YOU INTERESTED IN DOING YOUR OWN.
HI HI

FOR THE 100,000:1, I will only do these maps commercially, because the 3 days
worth of effort only results in a map that covers a few miles.  Only a few
HAMS will even live inside one of these small areas.  If you are a MAP-ology
expert, or an ace programmer and can help us figure out a way to better
automate the process, then we would love to make the whole process easier
for everyone.  Contact Ken Swaggart, W7KKE in Calif, or myself.

*****************************************************************************
README.100   (as of 2/13/94)

                   1:100,000 CD ROM Map Making

                            W7KKE
                         Ken Swaggart
               w7kke @ ka6eyh.#nocal.ca.usa

The 1:100,000 line segment maps contained on the USGS CD ROM uses
a different format than the 1:2,000,000 CD ROM maps. My first
utility, CONV100.BAS, converts the 1:100,000 map "Optional Format"
extracted files to the .GRF file format which can be read by the APRS
MAPFIX.BAS program. When you import these map files into MAPFIX you
will notice they are extremely fragmented into small line segments
which are not adjacent. The second utility, DEFRAG.BAS takes the saved
MAPFIX file and defragments it.

Here are the step by step instructions:

  1. Create a directory on your hard drive where you will be work
on your maps. Stick my programs in this directory and copy
MAPFIX.BAS from the APRS directory into this working map directory.
I use C:\APRSMAPS as my working map directory.

  2. Extract "Transportation" and "Hydrography"  for your area of
interest from the 1:100,000 CD ROM using the extraction program
which comes on the CD ROM. I zoom down to the 15' scale on the USGS
extraction program before I extract the map file. Copy these files
into your working map directory.

  3. Run CONV100.BAS.  The program will extract each feature, such
as Interstates, US Highways, etc., as separate files with the
filename extension of .GRF.

  4. Run MAPFIX.BAS. Import the first converted .GRF file  by using
the @U command in MAPFIX.BAS. After the file is imported you and on
your MAPFIX screen use the Join command (@J) to join adjacent line
segments for this feature. Save this feature as its own .MAP file
and repeat the process for each map feature you created.

  NOTE: Make sure you use the same origin and scale each time you
        import the .GRF file using the @U command.

  5. Run DEFRAG.BAS to de-fragment the .MAP files you have created.

     If DEFRAG.BAS crashes, you can recover most of your defragmented
     segments. Look for SCRATCH1.TMP and SCRATCH2.TMP. Rename the
     larger of these two and delete the remaining .TMP file.

  6. Use a text editor, such as DOS's EDIT, and examine your
defragmented .MAP files and tweak as you see fit. You may find that
some line segments consist of only one point - EDIT THOSE OUT

  7. Run MAPFIX.BAS again and import your defragmented .MAP file.
Check it over for any obvious errors, correct those, then run the
smoother function (@SMOOTH) to pare down the .MAP file.

  Note: I generally do not run smoother on the water boundaries and
streams, but hand edit to reduce the points. Another option which
works well is to use the U command to overlay the water boundaries
and streams, then manually draw in the rivers with the @new and @A
commands. This results if far fewer points than the automatic
method.

 8. Save your smoothed .MAP file and import your next feature and
repeat the process. (I only run the smoother on each separate
feature file so I can control what gets smoothed.) Sometimes I have
to bump the smoother factor way up to pare down the points to fit
the total point limit for APRS.

  9. Use a text editor, such as DOS's EDIT, to assemble the final
map from all the defragmented and smoothed .MAP files created for
each map feature type.  BOB COMMENTS:  I prefer to just use the
MAPFIX @IMPORT command to bring these into the working map...

  10. Run MAPFIX again and load the resulting complete .MAP file
with all your features. If it is too large, which it probably is,
delete points until you have a map which will fit within APRS
limits.

  - There is some small errors (less than 1/4 mile) toward the north east
and north west sections of the map. This is due to the conversion scheme
I use to get from the UTM grid to lat/long. The two grids do not line up
exactly. When you join two adjacent 15' maps, you will see that the roads
are off by about 100 meters. Use mapfix to tweak the lines so they join.


Please send me any program enhancements/fixes you make to either
program. I'm sure there are a lot better ways to defragment the
.MAP files - there MUST be!

PS - watch out for "Streams" - when I extract the ones around here
the resulting .MAP has thousands of points!

Changes from CONV100B to CONV100C.bas are:

   Now does not crash on very large street files. Before, it would often
   crash when extracting class 4 streets (neighborhood type streets). Be
   prepared for very large class 4 files - over two megabytes for urban areas.

   I extract the source date for the data on the 1:100,000 CD ROM. This is
   to clue you that you might be working with old data - some in northern
   California are 12 years old!


