[Loadstone] info request about default database files

Grzegorz Zlotowicz grzezlo at gmail.com
Thu Mar 31 16:11:42 BST 2011


Hi,
there is a method of creating binary LS databases using pc.
The process is a bit tricky to setup, but it works fine.
Having installed nokia sdk we need the python for s60 sdk, and a python
script to automate the process of importing database from text to binary
form - in fact it is rather simple program which executes database queries
in the symbian environment.
I'm running this script under python in the Nokia emulator, and
everything works well.
There is a way of running this emulator without gui window, so it can work
in background.
I developed a program to convert for Loadstone map data from polish map
project (called ump), which for area of poland is better than openstreetmap;
the process creates over 240 databases for different areas and poi sets, 
over 40 megabytes of compressed text, and over 100 megabytes of compressed 
binary databases.
The biggest database is all provinces and all pois for Poland, it consist at 
the moment of 626100 points and works well in Loadstone.
The binary database size is over 54 megabytes, source text file has size of 
over 46 megabytes (uncompressed ofcourse).
Try imagine how long it would take to import such file using even very fast 
phone...
100 hours? 25 hours? Maybe, who knows...
A nd how long it will take to generate this database using pc nokia 
emulator?
10 hours? 5 hours? No.
Only... 1533 Seconds, which is 25 minutes.
Whole process of generating mentioned 240 databases, converting to binary 
form, and compressing everything into zip archives takes about 170 minutes 
and during this process I can use the computer without noticing difficulties 
or slowdowns.
And this is the end of good news, the problem is, that installing of symbian 
sdk and python for sdk is not as simple as I'd like it to be, so this method 
is probably worth implementation for developers only, for generating huge 
amount of map data periodically.
To use symbian sdk user must register on forum nokia, and finding the proper 
file to download is not too simple as I remember.
The next question is the script license - mentioned Python script and its' 
launcher are developed by one of my colleagues for map conversion project, 
we don't talk about possibility or impossibility of giving this tool to 
other users, surely I'd have to ask before making it publicly available.
Greetings, Greg.
----- Original Message ----- 
From: "Shawn Kirkpatrick" <shawn at loadstone-gps.com>
To: <loadstone at loadstone-gps.com>
Sent: Wednesday, March 30, 2011 1:15 PM
Subject: Re: [Loadstone] info request about default database files


At this time there is no way to import files in to a loadstone database
using the pc.
If you load a file of the size you're talking about then it's likely
loadstone will become almost unusable. The maximum size of a file that still
gives reasonable response is around 100000 points or so. You'd be better off
to split a file of this size in to parts that make sence like cities. You
may also want to upload this file to the point share site and then users
could download selected parts of it as needed. This is the way to share map
data with the community, not as databases in the binary format.

On Wed, 30 Mar 2011, Ömer Yesiltas wrote:

> Hi, yes, that is the normal way  of doing it. But the file we have is
> large and it takes much time to import it.
> Since we will share a map of Turkey with the blind people here, I am
> looking a shorter and faster way to import files or making a default
> database file on the computer.
> Because loadstone crushes when we want to import files with 10-20mb in
> size. We tried dividing these files into several pieces but this time it
> crushed due to the default file size has reached a very large size.
> And, there is a remarkable slowing when we load default files more than
> 10mb or more. The search action takes much time and it sometimes crushes
> the program while searching.
> Another problem with importing etc is, loadstone responds very slow while
> importing and some talks functions like date and time and learn status do
> not work.
> .when the program crushes during import, or phone's battery empties etc,
> the default database file is being corrupted and we need to import all the
> data again.
> It sometimes takes hours, even days to import a file to the program and
> these are not old phones, these are faster new phones.
> www.cepdenizi.net
>  ----- Original Message -----
>  From: Jacob Kruger
>  To: loadstone at loadstone-gps.com
>  Sent: Wednesday, March 30, 2011 12:17 PM
>  Subject: Re: [Loadstone] info request about default database files
>
>
>  Ok, then if I wanted to replace the default database, I would just go
> delete the current one, since then a clean/empty one would be created
> automatically the next time you started loadstone, and that txt file would
> then be put into import export subfolder, where you would then import it
> into the new, clean default database.
>
>  Stay well
>
>  Jacob Kruger
>  Blind Biker
>  Skype: BlindZA
>  '...fate had broken his body, but not his spirit...'
>
>    ----- Original Message -----
>    From: Ömer Yesiltas
>    To: loadstone at loadstone-gps.com
>    Sent: Wednesday, March 30, 2011 10:51 AM
>    Subject: Re: [Loadstone] info request about default database files
>
>
>    Hello, are you sure? the file is actually not in default database
> format.
>    It is a file with txt extension and has been converted from a different
> map data. the file context is not similar to the defaults. the file
> context is like this:
>    #!lsdb
>    table,point
>    name,latitude,longitude,accuracy,satellites,priority,userid,id
>    "askeri bölge  istanbul
> (pendik)",408582200,292658200,1,0,0,58785,970782431
>    "evliyaullah tan seyh kemikli hazretleri türbesi güzelyali mah pendik
> istan",408595300,292751300,1,0,0,58785,970782672
>    "sahil dürüm evi güzelyali mah pendik istanbul
> ",408567800,292771900,1,0,0,58785,970782673
>    "kugum pasta q güzelyali mah pendik istanbul
> ",408581500,292799400,1,0,0,58785,970782674
>    "otel sidemo güzelyali mah pendik istanbul
> ",408581500,292717000,1,0,0,58785,970782675
>    and goes like this.
>    now, can we just use this file just putting it into database folder by
> renaming it  as default?
>    www.cepdenizi.net
>      ----- Original Message -----
>      From: Jacob Kruger
>      To: loadstone at loadstone-gps.com
>      Sent: Wednesday, March 30, 2011 7:52 AM
>      Subject: Re: [Loadstone] info request about default database files
>
>
>      AFAIK, all you'd really need to do is rename the file to DEFAULT, and
> place it in the databases subfolder for loadstone, and there you go...
>
>      Stay well
>
>      Jacob Kruger
>      Blind Biker
>      Skype: BlindZA
>      '...fate had broken his body, but not his spirit...'
>
>        ----- Original Message -----
>        From: Ömer Yesiltas
>        To: loadstone at loadstone-gps.com
>        Sent: Wednesday, March 30, 2011 12:29 AM
>        Subject: [Loadstone] info request about default database files
>
>
>        Hello, we have a great map converted to loadstone format but we did
> not make it default. Is there a computer program to convert these txt
> files to loadstone default format?
>        Secondly, when the database importing aborts in any case, the
> default file is corrupted. is there a way to repair these files or is
> there any way to continue importing after abort?
>        www.cepdenizi.net
>
>
> ------------------------------------------------------------------------
>
>
>        _______________________________________________
>        Loadstone mailing list
>        Loadstone at loadstone-gps.com
>        http://www.loadstone-gps.com/mailman/listinfo/loadstone
>
>
>
> --------------------------------------------------------------------------
>
>
>      _______________________________________________
>      Loadstone mailing list
>      Loadstone at loadstone-gps.com
>      http://www.loadstone-gps.com/mailman/listinfo/loadstone
>
>
>
> ----------------------------------------------------------------------------
>
>
>    _______________________________________________
>    Loadstone mailing list
>    Loadstone at loadstone-gps.com
>    http://www.loadstone-gps.com/mailman/listinfo/loadstone
>
>
>
> ------------------------------------------------------------------------------
>
>
>  _______________________________________________
>  Loadstone mailing list
>  Loadstone at loadstone-gps.com
>  http://www.loadstone-gps.com/mailman/listinfo/loadstone
>


--------------------------------------------------------------------------------


> _______________________________________________
> Loadstone mailing list
> Loadstone at loadstone-gps.com
> http://www.loadstone-gps.com/mailman/listinfo/loadstone
>



More information about the Loadstone mailing list