Spatial Data Connection in Mapguide

this is my findings during my research of publishing marine physical oceanography of web using Mapguide.

here's the thing. it is so hard to load a geodatabase (.mdb) perfectly into mapguide. mapguide only able to support shapefile (via load data procedure) in default. however, the aims of our research is to create a centralized database, not a file-based data compilation on a web server.

here's the research that i've done in a way to achieve the objective of publishing centralized database on the web using open source mapguide.

mdb file actually can be loaded into mapguide using a little customization of the software. all you need is OGR provider that can be downloaded in OSGEO website..

extract the file and paste the .dll file into opensource FDO directory that usually found in:

c:/program files/osgeo/bin/fdo

but, this provider is still need to be revised since it only load spatial data (with geometry properties). in a normal centralized database, not all data are spatially georeferenced. supported non-spatial data such as land parcel owner is still needed to create a meaningful database!

ok, here's the summary of my finding.

Data format	Provider used	Findings
*.mdb	OGR Provider	Only load spatial data. Non-spatial data that implemented in the database is not included during data loading.
*.mdb	ODBC Provider with system DSN setting	Load all data in the geodatabase, but it neglect the geometry (spatial) properties of the data
*.shp	default	Load the Shapefile data, but doesn’t has ability to load external non spatial data (*.dbf file)
postgis	Postgis Provider	Data need to be converted into postgis format using FME. The spatial extend need to be specified to view the geometry of the data. No issues of spatial – non spatial table.
*.kml	KML FDO Provider	Installed the .dll file, but it is still not appear in the menu of data connection. Duh!!

niu werk

its been a long time since my last update..

however,my data model was implemented n now we're working on publishing the thingy on internet server...

actually its nabil's work... but he's quitting n i have to do every single thing, including hadi's ontology....

i really hope that Rosddi can come out with useful data catalogue since it will use it in my annotation and ontology.

ontology is a semantic, synonymous, antonymous, and meronim database that related to certain keyword that exist in MyNODC database.

in other hand, annotation is the evidence. evidence of the relationship, or in this case, the evidence of relationship between the keyword with related words. the evidence is in form of trusted journal, organization and formal documentation...

cruise

A Customization of the Arc Marine Data

Model to Support Whale Tracking via

Satellite Telemetry (Brett Lord Castillo)

The Operations group is divided into two areas,

Cruise and Approaches (Figure 7).

Cruise involves a small number of generic object classes to link field observations to the

person making the observation.

Approaches handle the specific operational situation of

approaching an animal and deploying a tag.

Cruise is essentially a customization of the Arc Marine SurveyInfo object class tables.

SurveyInfo links an InstantaneousPoint to a unique survey operation.

This point may

represent a sighting, photograph, deployment, telemetry location, or a wide variety of

other features.

When this point is linked to a survey though, that survey has a specific

crew, identified by CrewKey, and specific crew members in that crew, identified by the

Crew class object.

Thus, a crew has crew members and carries out one unique survey.

SurveyInfo is also a dimension of the ApproachEvent, a linking dimension table for the

Approach object group.

IHO s-100

Goals for S-100

S-100 will support items such as imagery and gridded data, 3D and time-varying data (x, y, z, and time), and new applications that go beyond the scope of traditional hydrography; for example, high-density bathymetry, sea floor classification and marine GIS. It will also enable the use of web-based services for acquiring, processing, analysing, accessing and presenting data. It is important to recognise that S-100 is not an incremental revision of the current Edition 3.1 of S-57. S-100 will be a new standard that includes additional content and support of new data exchange formats.
S-100 was released as a draft version in February 2008 and is in its development phase.

why Visio 2003?

Microsoft has not made an UML to XMI

export utility available for Visio 2007. Thus, Visio 2007 cannot be used at

this time to generate modified database schemas for use by ESRI CASE

tools.

example of marine database (web based)

http://www.pimrisportal.org/index.php?option=com_content&view=article&id=73:international-maps&catid=60:maps-and-gis&Itemid=68

Seamount Catalog The Seamount Catalog is a digital archive for bathymetric seamount maps that can be viewed and downloaded in various formats. This catalog contains morphological data, sample information, related grid and multibeam data files, as well as user-contributed files that all can be downloaded. Currently this catalog contains more than 1,800 seamounts from all the oceans.

http://earthref.org/cgi-bin/er.cgi?s=sc-s0-main.cgi

global information system for coral reefs

http://reefgis.reefbase.org/

case study : arcmarine data model

ni case study utk arcmarine data model..

so mcm boleh guide utk datamodel testing.

http://dusk.geo.orst.edu/esri04/p1458_alli.html

erm.. nnt kena bace n rephrase blk lah..

methodologies,

Modeling methodologies

Data models represent information areas of interest. While there are many ways to create data models, according to Len Silverston (1997)^[6] only two modeling methodologies stand out, top-down and bottom-up:

• Bottom-up models are often the result of a reengineering effort. They usually start with existing data structures forms, fields on application screens, or reports. These models are usually physical, application-specific, and incomplete from an enterprise perspective. They may not promote data sharing, especially if they are built without reference to other parts of the organization.^[6]
• Top-down logical data models, on the other hand, are created in an abstract way by getting information from people who know the subject area. A system may not implement all the entities in a logical model, but the model serves as a reference point or template.^[6]

Sometimes models are created in a mixture of the two methods: by considering the data needs and structure of an application and by consistently referencing a subject-area model. Unfortunately, in many environments the distinction between a logical data model and a physical data model is blurred. In addition, some CASE tools don’t make a distinction between logical and physical data models.^[6]

^{reeingineering nmpk mcm xbagus utk SDI.. kn?}

^{sbb xpromote data sharing, specific n etc.}

^{so ape method nk gune ni?}

warehouse. data mart. clearinghouse. SDI.

definition and explaination of SDI, clearinghouse, warehouse, n etc:

http://www.geom.unimelb.edu.au/research/publications/IPW/SDIDefinitionsAusSurv.html

tidak ilmiah

hurm.
aku serabut.
mana mungkin data model jd se-complicated ni.
owh, maybe sbb nama dia superdatamodel..
its get confusing more n more as i read.

kenape mcm ni?

aku cuba utk cantum each part of my pembacaan n exploration to form a data model,

tapi still, aku xdpt nk cari mane logik nye.

mane korelasi data model dgn main2 software image processing.

data model ade fungsi me"revert" kah?

setahu aku data model tu cuma satu abstraction data n relationship nye dalam satu database.

tu termasuklah ciri-ciri object oriented macam inheritance, encapsulation n etc.

kalu logik pon xdpt aku temukan, macam mane aku create something new?

how?

aku xtaw nk buat ape dah.

kenapa kita cakap dalam bahasa yg sama,

tapi aku xdpt kefahaman yg sama dari ape yg dr razib maksud kn?

arini dah hari khamis.

n aku still clueless.

i want to be good so bad.

tp mcm2 halangan ada.

aku blaja dlu xberkat ke??

ke aku byk sgt dosa?

i need someone to communicate with.

org yg phm GIS and its nature.

bkn mcm dr razib. dia xphm aku ckp. n aku pon xphm ape dia maksudkan.

aku suke keje cni. but i dun think bosses rase puas hati bayar aku rm2000 for nothing.

today activity (x produktif sgt)

http://coralreefwatch.noaa.gov/satellite/methodology/methodology.html

pasal ape link atas tu?

owh.. related to coral reef thingy.

A-Z bout coral bleaching n SST concept.

what did i do today?

1- ask dr razib bout "am i on the right lane" related to SST data processing.

apparently im not manyimpang yet.

2- find software - ILWIS

download n install n explore

3- looking for marine satellite data, but sadly, xde pon dekat net.

dats all. now sedang explore2 according to tutorial dia.

tonight nk explore lagayyyyy!

SST algorithm

*refer blue book for previous topic for SST processing

i dun think i manage to understand those formulas, however, i'll include it in journal write up.

reference:

http://www.bom.gov.au/sat/SST/sst.shtml

McClain, E. P, et al., 1984, 'Comparative Performance of AVHRR-Based Multichannel Sea Surface Temperatures', J. Geophys. Res., 90, pp11587-11601

NOAA Polar Orbiter Data User's Guide

• http://www2.ncdc.noaa.gov/docs/podug/index.htm

NOAA KLM USER'S GUIDE

• http://www2.ncdc.noaa.gov/docs/klm/

NESDIS SST Equations

• http://manati.orbit.nesdis.noaa.gov/sst/cwIntroduction.html

Links valid at: 26th July 2007

SST Algorithms

If the solar zenith angle is less than or equal to 75° then the day time algorithm is used. Otherwise the night time algorithm is used.

The mean of 3 separate multi channel algorithms are used to compute the SST at night time and three algorithms must be within 2 °C otherwise the pixel is rejected. Only one algorithm is used during the day as channel 3 contains reflected sunlight and can not be used.

Daytime Algorithm

• check satellite zenith angle is less than 53°
• do land/ sea test
• check solar zenith angle - do not process if less than 1°
• gross IR test - if the channel 4 temperature is less than -5° C then do not compute a SST
• visible cloud threshold test - if the corrected albedo (albedo value divided by the cosine of the solar zenith angle) is greater than 10 percent then do not compute a SST
• visible vegetation threshold test - if the vegetation index (corrected channel 2 albedo divided by corrected channel 1 albedo) is greater than 0.75, do not compute a SST
• visible uniformity test - the corrected channel 2 albedos for all pixels in a three by three pixel box centred on the target pixel must be within 0.32 percent of the median value for the box. The maximum and minimum values must be within 0.64.
• daytime IR cloud test - if a calculated channel 4 temperature based on the channel 5 value (channel 5 temp * 1.0439 - 11.49) differs from the actual channel 4 temperature by more than 1.0° C then do not compute a SST. This test is not used by McClain et al. (1984) for daytime data but has been used historically at the BoM. The band widths for NOAA-11, 12, 14 and 15 channels 4 and 5 are the same on each satellite so the coefficients for this equation remain constant.

Multi channel SST calculation

• NOAA-11
  

CPSST Day Split Window Algorithm

sst = (0.19069 * T₅ - 49.16) / (0.20524 * T₅ - 0.17334 * T₄ - 6.78) *
(T₄ - T₅ + 0.7890) + 0.92912 * T₅ + 0.81 * (T₄ - T₅ ) * (sec(ZA) - 1) + 18.98

• NOAA-12
  

MCSST Day Split Window Algorithm

sst = (1.008574 * T₄ ) + 2.452585 * (T₄ - T₅ ) +
0.823990 * (T₄ - T₅ ) * (sec(ZA) - 1) -275.717 + 273.16

• NOAA-14
  

MCSST Day Split Window Algorithm

sst = (1.017342 * T₄ ) + 2.139588 * (T₄ - T₅ ) +
0.779706 * (T₄ - T₅ ) * (sec(ZA) - 1) + -278.43 + 273.16

• NOAA-15
  

MCSST Day Split Window Algorithm

sst = (0.959456 * T₄ ) + 2.663580 * (T₄ - T₅ ) +
0.570613 * (T₄ - T₅ ) * (sec(ZA) - 1) + -261.03 + 273.16

where:

sst - computed SST value in degrees (°) C.

T₄ - channel 4 scene temperature

T₅ - channel 5 scene temperature

ZA - solar zenith angle

Algorithms for NOAA-12, 14 and 15 posted on NOAA/ NESDIS web server. The NOAA-11 algorithm is historical and is no longer operational.

Night time Algorithm

• check satellite zenith angle is less than 53°
• do land/ sea test
• check solar zenith angle, if greater than 75° and if channel 2 reflectance is less than one percent, then use night-time algorithm. If the reflectance is greater than one percent, do not compute a SST
• gross IR test - if the channel 4 temperature is less than -5° C then do not compute a SST
• IR uniformity test - the channel 4 temperatures for all pixels in a three by three pixel box centred on the target pixel must be within 0.2° C of the median value for the box. The maximum and minimum values must be within 0.4° C
• nighttime IR cloud test - if a calculated channel 4 temperature based on the channel 5 value (channel 5 temp * 1.0439 - 11.49) differs from the actual channel 4 temperature by more than 1.0° C then do not compute a SST.
• night time low stratus cloud test - the difference obtained when subtracting the channel 3 temperature from the channel 5 temperature must be less than or equal to -0.6° C.
• Multi channel SST calculation. Three separate algorithms are used. The computed values for the three algorithms must all agree within 2.0° C.

• NOAA-11

MCSST Night Dual Channel Algorithm

sst1 = (0.17079 * T₄ - 58.47) /
(0.17334 * T₄ - 0.07747 * T₃ - 33.74) *
(T₃ - T₄ - 6.440) + 0.98530 * T₄ + 1.97 * (sec(ZA) - 1) + 15.88

MCSST Night Split Window Algorithm

sst2 = (0.19596 * T₅ - 48.61) /
(0.20254 * T₅ - 0.17334 * T₄ - 6.11) *
(T₄ - T₅ + 1.4600) + 0.95476 * T₅ + 0.98 * (T₄ - T₅ ) * (sec(ZA) - 1) + 9.32

MCSST Night Triple Channel Algorithm

sst3 = (0.16835 * T₄ - 34.32) /
(0.20524 * T₅ - 0.07747 * T₃ - 20.01) *
(T₃ - T₅ + 14.86) + 0.97120 * T₄ + 1.87 * (sec(ZA) - 1) - 3.43

• NOAA-12

MCSST Night Dual Channel Algorithm

sst1 = (1.017736 * T₃ ) + 0.426593 * (T₃ - T₄ ) +
1.800916 * (sec(ZA) - 1) - 276.264 + 273.16

MCSST Night Split Window Algorithm

sst2 = (1.013674 * T₄ ) + 2.443474 * (T₄ - T₅ ) +
0.314312 * (T₄ - T₅ ) * (sec(ZA) - 1) - 277.797 + 273.16

MCSST Night Triple Channel Algorithm

sst3 = (1.003194 * T₄ ) + 1.007171 * (T₃ - T₅ ) +
1.174698 * (sec(ZA) - 1) - 273.262 + 273.16

• NOAA-14
  

MCSST Night Dual Channel Algorithm

sst1 = (1.008751 * T₄ ) + 1.409936 * (T₃ - T₄ ) +
1.975581 * (sec(ZA) - 1) - 273.914 + 273.16

MCSST Night Split Window Algorithm

sst2 = (1.029088 * T₄ ) + 2.275385 * (T₄ - T₅ ) +
0.752567 * (T₄ - T₅ ) * (sec(ZA) - 1) - 282.24 + 273.16

MCSST Night Triple Channel Algorithm

sst3 = (1.010037 * T₄ ) + 0.920822 * (T₃ - T₅ ) +
0.067026 * (sec(ZA) - 1) - 275.364 + 273.16

• NOAA-15

MCSST Night Dual Channel Algorithm

sst1 = (1.041037 * T₄ ) + 1.587582 * (T₃ - T₄ ) +
1.677430 * (sec(ZA) - 1) - 283.51 + 273.16

MCSST Night Split Window Algorithm

sst2 = (0.993892 * T₄ ) + 2.752347 * (T₄ - T₅ ) +
0.662999 * (T₄ - T₅ ) * (sec(ZA) - 1) - 271.40 + 273.16

MCSST Night Triple Channel Algorithm

sst3 = (1.015354 * T₄ ) + 1.063572 * (T₃ - T₅ ) +
1.294955 * (sec(ZA) - 1) - 276.76 + 273.16

where:

sst n - computed SST value in degrees ° C.

T₃ - channel 3 scene temperature

T₄ - channel 4 scene temperature

T₅ - channel 5 scene temperature

ZA - solar zenith angle

Algorithms for NOAA-12, 14 and 15 posted on NOAA/ NESDIS web server. The NOAA-11 algorithm is historical and is non-operational.

• the SST value used is the mean of the three values computed.

Climatology test

Computed SST rejected if differs from climatology by more than 10°

SST and coral bleaching

update for yesterday n today.

------------------------------

yesterday i do some background reading on coral bleaching and SST.

i understand the concept of both case study..

but what spinning me around is how does the SST data extracted?

i mean, the data processing.

what is the raw data for SST?

are the colourful classified satelite images is the product of combination of SST observation from satellites and thermometer ?(since there is various type of SST and sources)

or is there any satellite that can detect the temperature without needing any processing process?

after long days of reading (plus sleepy cuz fasting)

i find nothing except the example of applied arc marine data model (which solve my one entangled misunderstanding issue)

ok, today i found 1 article from australia govt which give answers for my unpleasant understanding of SST.

this is the link: http://www.bom.gov.au/sat/SST/sst.shtml

and this is part of it (which makes me write this update -->improvement)

Sea Surface Temperatures (SSTs) maps derived from remote sensing by satellites have been available since the 1970s. The Bureau currently uses measurements from the Advanced Very High Resolution Radiometer (AVHRR) on board the National Oceanic and Atmospheric Administration NOAA series of polar orbiting satellites to derive SSTs for the Australian region. The data is calibrated and quality controlled against SST data collected from ships and drifting buoys. The SSTs are used in real time operations and also archived as the data as part of Australia's National Climate Record.

ok, since i have the answer for my dilemma, now i can proceed to search info about the data processing (procedure and software) and find some public data + open source software!

c u l8r.

coral bleaching is like broke-up thingy (which leads to self-destroy effect) between coral and symbiotic algae. i wrote it down on blue book!

17-7-2010 meeting

ok. last saturday, we're having our weekly meeting.

yeah, mine was quite sucks.

hadi n nabil doin jz fine.

they hardly understand my explaintation.

i dont think my method is too hard to understand.

dr hisham is d only one who understands the method that i used.

however, i still have to change it. (my degree of stubborn-ness was reduced)

im creating super data model, which is mother of all data model that related in marine SDI db.

so my super data model supposely encompassed all the characteristics of related data model.

for start, lets focus on our case study - SST and coral bleaching.

Dr also asked me to download public data, few opensource software (off course it hv to relate with d case study) play with it and came out with d data model within.

i understand d concept, but im not sure how to do it,

since both of case study involve satellite images.

by hook or by crook, i hv to do it good. i hv to b good.

hadi suggest me to explore those software (4 image processing)

-GRASS

-ILWIS

-other software published on OSGeo.org

come on jen! u can do it! u r a superwoman who create superdatamodel! SuckItUp!

something to think of.

we must extend the standard point, line and polygon representation of of geographic features to meet the volumetric and temporally dynamic nature of marine environment.

a generic data model was needed to meet the core challenges of designing a more temporally dynamic and volumetric representation of marine features.

the core idea of arcmarine was to develop common data types as core building blocks for the development of specific feature classes for coastal and marine application. this common data type needed to be broad and comprehensive.

amazed by esri arcmarine data model.

ok.. finally i manage to read 1 section of Arcmarine data model (as literature review for my DM) and i already amazed. Since it is so dynamic n temporal-able. Image above (plz focus on highlighted part) show (more or less) the ability of that layer and data model to show/display/model the tidal variance to describe the shoreline! i was like - gila lah! database gile temporally dynamic smpi tahap boleh represent pasang surut air laut (sbb tu shoreline x tetap). tabik spring lah esri!!

so, the question is - mcm mane nk implement bnde tu? think suzanna. think.

some idea on data warehouse design

http://it.toolbox.com/blogs/dw-cents/data-warehouse-data-model-design-18699

A friend of mine recently asked me to help her develop some ideas for explaining data warehousing to a group of business users. It seems there was some confusion in her company regarding why their so-called data warehouse was really just a data dumping ground and not a true best practices data warehouse. The following list details what I think separates a data warehouse from an archive database, data mart, ODS, etc. The data warehouse data model should be designed for the following:

1) Data is comprehensive - Data is captured and consolidated from multiple systems.

2) Data is conformed - This is the famous line "single version of the truth". Data elements are conformed so that the definitions of "customer" or "revenue" mean the same thing no matter which system it originated. Tables are conformed when they can be queried across dimensions and facts without changing the meaning of the results. This is what is needed to truly integrate data in the warehouse.

3) Data is granular - Ideally, we capture and store data at its lowest level of granularity. You can always aggregate up, but you can't drill down if the data isn't stored that way.

4) Data is historical - The data warehouse is able to present a view of the business at a particular point in time and track Key Performance Indicators (KPI's) over time.

5) Data is shared - A data warehouse that cannot be queried or otherwise accessed has little value.

biz model

wiki says:

A business model describes the rationale of how an organization creates, delivers, and captures value^[1] - economic, social, or other forms of value. The process of business model design is part of business strategy.

In theory and practice the term business model is used for a broad range of informal and formal descriptions to represent core aspects of a business, including purpose, offerings, strategies, infrastructure, organizational structures, trading practices, and operational processes and policies

i say:

Marine SDI promote data sharing using data custodian + policies and standard is biz model for NODC.

the workflow on how data obtained, processed n all pathwayc(data retrieval from db using annotation & ontology & SBML) is d biz model. is it true?

further searching :

Business process modeling (BPM) in systems engineering and software engineering is the activity of representing processes of an enterprise, so that the current process may be analyzed and improved. BPM is typically performed by business analysts and managers who are seeking to improve process efficiency and quality. The process improvements identified by BPM may or may not requireInformation Technology involvement, although that is a common driver for the need to model a business process, by creating a process master.

modeling method 1

http://www.learndatamodeling.com/

http://en.wikipedia.org/wiki/Data_modeling

Modeling methodologies

Data models represent information areas of interest. While there are many ways to create data models, according to Len Silverston (1997)[6] only two modeling methodologies stand out, top-down and bottom-up:

Bottom-up models are often the result of a reengineering effort. They usually start with existing data structures forms, fields on application screens, or reports. These models are usually physical, application-specific, and incomplete from an enterprise perspective. They may not promote data sharing, especially if they are built without reference to other parts of the organization.[6]

Top-down logical data models, on the other hand, are created in an abstract way by getting information from people who know the subject area. A system may not implement all the entities in a logical model, but the model serves as a reference point or template.[6]
Sometimes models are created in a mixture of the two methods: by considering the data needs and structure of an application and by consistently referencing a subject-area model. Unfortunately, in many environments the distinction between a logical data model and a physical data model is blurred. In addition, some CASE tools don’t make a distinction between logical and physical data models.[6]

13-7-2010

dear avatar logblogvlog..
ahaha...
today in didnt do any progress on my data model.
hadi assign me to do the tender bidding thingy that MUST be submitted 2 days from now.
its so wordy n lengthy.
i hate talking about server.

12-7-2010

today i finish my whole day modifying E-cost's manual. pfft.
now at 7.23 pm, im still at office.
im so exhausted, i dunno whether i manage to do some reading tonight.

logblog

owh ya..
i have to document my everyday works n progress in this log-blog.
i love to write, but not necessary an educational type of write up.
ill try my best.
hope my part of data modelling will be a good contribution to my research group.
*finger-cross*