Re: [OpenMap Users] reading very large shapefiles/dbf files

From: C S <usmsci_at_yahoo.com>
Date: Mon, 4 Oct 2010 11:45:08 -0700 (PDT)

Don,

  thanks for the reply. I am glad that you mentioned the SpatialIndex because reading every shapefile index is exactly what i don't want to do. This will also alleviate having to read(i hope) or stream an entire dbf file. Am i correct in assuming that i can only siphon out dbf records that are of interest to me rather than streaming the whole dbf file?

I am going to practice with some of these classes and might have some follow up questions for you, if you dont mind, if there are things that dont make sense. About the SpatialIndex though, you mentioned that i can only return shapefile records that lie within a bounding box. Does this also work for shapes that arent a perfect square/rectangle?(multiple points, irregular shapes), because this is a target area for me to support.

thanks again for your time....

Cws(Chad)

--- On Mon, 10/4/10, Don Dietrick <dfdietrick_at_gmail.com> wrote:

> From: Don Dietrick <dfdietrick_at_gmail.com>
> Subject: Re: [OpenMap Users] reading very large shapefiles/dbf files
> To: "C S" <usmsci_at_yahoo.com>
> Date: Monday, October 4, 2010, 1:13 PM
> Hi cws,
>
> You can do this, using the EsriGraphicFactory and the
> DbfFile classes.
> The EsriGraphicFactory has methods that let you read from
> a shp file
> one record at a time.  You'll have to manage the byte
> offsets, to
> manage where the factory reads in the file for the next
> record.  The
> getEsriGraphics(...) method source code can show you how to
> do that.
> If you want to be able to sort through the shp file for
> only shapes
> that lie within a bounding box, check out the SpatialIndex
> class.
> It's what the ShapeLayer uses to control how the shape file
> is read.
> You can create a spatial index from the shape file using
> that class.
>
> The DbfFile class lets you retrieve records from the dbf
> file - so you
> can either read that file, look at the attributes and then
> determine
> which shapes to get based on those attributes, or you can
> use the
> spatial index to do your filtering spatially before
> bothering to re
this helps,
>
> Don
>
>
> On Mon, Oct 4, 2010 at 12:24 PM, C S <usmsci_at_yahoo.com>
> wrote:
> > Hi all,
> >
> >   i have some code currently, that reads in
> shapefiles/dbf files using the openmap API. I have ran into
> a bottleneck now reading very large shapefiles. The system
> that is reading the shapefiles is using web services in java
> and it doesnt have enough memory to load and stream in
> multiple files at once and sometimes even one very very
> large file.
> >
> > I am currently using the below code to read in the
> files:
> >
> > ShapeFile shapeFile = null;
> > DbfInputStream dbfFile = null;
> > DbfTableModel dbfTable = null;
> >
> > .....
> > .....
> >
> > shapeFile = new ShapeFile(shpInfile);
> > dbfFile = new DbfInputStream(dbfInputStream);
> > dbfTable = new DbfTableModel(dbfFile);
> >
> > is there a better way to read in the shapefiles and/or
> dbf blocks at a time to preserve memory at run time? My
> thought is, is that i can read a section in memory, do some
> calculations, read the next block and write over the
> previous block of memory with new data and so on.
> >
> > Any help please on this matter is appreciated.
> thanks!
> >
> > -cws
> >
> >
> >
> >
> > --
> > [To unsubscribe to this list send an email to "majdart_at_bbn.com"
> > with the following text in the BODY of the message
> "unsubscribe openmap-users"]
> >
>


      

--
[To unsubscribe to this list send an email to "majdart_at_bbn.com"
with the following text in the BODY of the message "unsubscribe openmap-users"]
Received on Mon Oct 04 2010 - 14:45:52 EDT

This archive was generated by hypermail 2.3.0 : Tue Mar 28 2017 - 23:25:09 EDT