Missing Element with Grib 2 files #13

silv2015 · 2015-10-27T19:31:01Z

Hi!

First I want to tell you that this project looks very promising! Good Work!

In our project, we are already use GDAL and it's C# wrapper but we are looking for another library to load our grib 1 and 2 files.

I have a Grib 2 files (GFS.grb2) in which I have 4 elements that we call Channels:
Temperature
U component of Winds
V component of winds
Total Precipitation

I can read all channels except V component of winds.

It does work for all my Grib 1 files.

I cannot attach any files, but if you need more info (except I cannot send you the grib file!) please let me know!

If we can load all our Grib files with your library, we can make the switch and it will simplify a lot our work here! :)

0x1mason · 2015-10-28T15:48:00Z

@silv2015 Thanks for trying the library! I will look into this and see if I can reproduce the issue.

I just released a new version, though I have no reason to think it will solve your problem.

0x1mason · 2015-10-28T16:57:22Z

This works for me using v0.6.0:

            using(GribFile file = new GribFile(@".\TestData\Pacific.wind.7days.grb"))
            {
                var vComp = file.Where(m => m.Name.Contains("V V-component of wind m s**-1")).First();

                foreach (var val in vComp.GeoSpatialValues)
                {
                    Console.WriteLine("Lat: {0} Lon: {1} Val: {2}", val.Latitude, val.Longitude, val.Value);
                }
            }

Here's the file I used.

0x1mason · 2015-10-28T17:20:44Z

If that ^ doesn't work for you, can you explain what you mean by "doesn't work"? Are you getting an exception? If so, what does the exception say?

silv2015 · 2015-10-28T18:34:37Z

Hi 0x1mason!

Thanks for the answer, I downloaded your file and will give it a try

I will have another try with my file and send you what I mean by 'doesn't work'

it does not give me any exception, it's just kind of skip it.

Thanks again

silv2015 · 2015-10-28T18:57:43Z

Here I have the result...

If I load your file, yes it works:
0:[u] "U U-component of wind m s**-1" (instant):regular_ll:surface 1:fcst time 6 hrs :from 2015-09-02 1200
1:[v] "V V-component of wind m s**-1" (instant):regular_ll:surface 1:fcst time 6 hrs :from 2015-09-02 1200

But If I load my file (.grb2) here what I have:
0:[2t] "Temperature" (instant):regular_ll:heightAboveGround 2:fcst time 0 hrs :from 2015-10-26 1200
1:[10u] "u-component of wind" (instant):regular_ll:heightAboveGround 10:fcst time 0 hrs :from 2015-10-26 1200
2:[tp] "Total precipitation" (accum):regular_ll:surface 0:fcst time 99-102 hrs :from 2015-10-26 1200
3:[2t] "Temperature" (instant):regular_ll:heightAboveGround 2:fcst time 102 hrs :from 2015-10-26 1200
4:[10u] "u-component of wind" (instant):regular_ll:heightAboveGround 10:fcst time 102 hrs :from 2015-10-26 1200
5:[tp] "Total precipitation" (accum):regular_ll:surface 0:fcst time 102-105 hrs :from 2015-10-26 1200

I should have something for the V component of winds

But like I said, if I take any .grb (not .grb2), it works

Could it be related to Grib2 file?

0x1mason · 2015-10-28T19:20:08Z

You could be right, though I thought the file I used was GRIB 2, despite
the extension.

Can you see the v-component messages in GDAL, degrib, etc? I'm sure you've
confirmed it already, but worth asking.

On Wed, Oct 28, 2015 at 2:57 PM, Martin Brousseau notifications@github.com
wrote:

Here I have the result...

If I load your file, yes it works:
0:[u] "U U-component of wind m s
-1" (instant):regular_ll:surface 1:fcst time 6 hrs :from 2015-09-02 1200
1:[v] "V V-component of wind m s-1" (instant):regular_ll:surface 1:fcst
time 6 hrs :from 2015-09-02 1200

But If I load my file (.grb2) here what I have:
0:[2t] "Temperature" (instant):regular_ll:heightAboveGround 2:fcst time 0
hrs :from 2015-10-26 1200
1:[10u] "u-component of wind" (instant):regular_ll:heightAboveGround
10:fcst time 0 hrs :from 2015-10-26 1200
2:[tp] "Total precipitation" (accum):regular_ll:surface 0:fcst time 99-102
hrs :from 2015-10-26 1200
3:[2t] "Temperature" (instant):regular_ll:heightAboveGround 2:fcst time
102 hrs :from 2015-10-26 1200
4:[10u] "u-component of wind" (instant):regular_ll:heightAboveGround
10:fcst time 102 hrs :from 2015-10-26 1200
5:[tp] "Total precipitation" (accum):regular_ll:surface 0:fcst time
102-105 hrs :from 2015-10-26 1200

I should have something for the V component of winds

But like I said, if I take any .grb (not .grb2), it works

Could it be related to Grib2 file?

—
Reply to this email directly or view it on GitHub
#13 (comment)
.

silv2015 · 2015-10-29T12:43:12Z

Hi 0x1mason!

I can see the component with GDAL no problem, here is an output I copy from a test:

    [0] "GRIB_COMMENT=Temperature [K]"  string
    [1] "GRIB_ELEMENT=TMP"  string
    [2] "GRIB_FORECAST_SECONDS=0 sec"   string
    [3] "GRIB_PDS_PDTN=0"   string
    [4] "GRIB_PDS_TEMPLATE_NUMBERS=0 0 2 0 81 0 0 0 1 0 0 0 0 103 0 0 0 0 2 255 0 0 0 0 0"  string
    [5] "GRIB_REF_TIME=  1445860800 sec UTC"    string
    [6] "GRIB_SHORT_NAME=2-HTGL"    string
    [7] "GRIB_UNIT=[K]" string
    [8] "GRIB_VALID_TIME=  1445860800 sec UTC"  string

    [0] "GRIB_COMMENT=u-component of wind [m/s]"    string
    [1] "GRIB_ELEMENT=UGRD" string
    [2] "GRIB_FORECAST_SECONDS=0 sec"   string
    [3] "GRIB_REF_TIME=  1445860800 sec UTC"    string
    [4] "GRIB_SHORT_NAME=10-HTGL"   string
    [5] "GRIB_UNIT=[m/s]"   string
    [6] "GRIB_VALID_TIME=  1445860800 sec UTC"  string

    [0] "GRIB_COMMENT=v-component of wind [m/s]"    string
    [1] "GRIB_ELEMENT=VGRD" string
    [2] "GRIB_FORECAST_SECONDS=0 sec"   string
    [3] "GRIB_REF_TIME=  1445860800 sec UTC"    string
    [4] "GRIB_SHORT_NAME=10-HTGL"   string
    [5] "GRIB_UNIT=[m/s]"   string
    [6] "GRIB_VALID_TIME=  1445860800 sec UTC"  string

    [0] "GRIB_COMMENT=03 hr Total precipitation [kg/(m^2)]" string
    [1] "GRIB_ELEMENT=APCP03"   string
    [2] "GRIB_FORECAST_SECONDS=356400 sec"  string
    [3] "GRIB_REF_TIME=  1445860800 sec UTC"    string
    [4] "GRIB_SHORT_NAME=0-SFC" string
    [5] "GRIB_UNIT=[kg/(m^2)]"  string
    [6] "GRIB_VALID_TIME=  1446228000 sec UTC"  string

    [0] "GRIB_COMMENT=Temperature [K]"  string
    [1] "GRIB_ELEMENT=TMP"  string
    [2] "GRIB_FORECAST_SECONDS=367200 sec"  string
    [3] "GRIB_REF_TIME=  1445860800 sec UTC"    string
    [4] "GRIB_SHORT_NAME=2-HTGL"    string
    [5] "GRIB_UNIT=[K]" string
    [6] "GRIB_VALID_TIME=  1446228000 sec UTC"  string

0x1mason · 2015-10-29T15:11:06Z

Wow. Weird.

As a sanity check, I downloaded a GRIB2 file from NOAA and could see the
v-comp, so I haven't been able to repro the issue yet. Unfortunately, I'm
not going to be able to debug the issue until I can reproduce it. Here
are some options for moving forward:

You can find a shareable file that elicits the same behavior.
I can work something out with your employer in terms of signing an NDA.
You can send me the file privately and I will give you my word never to
share it with anyone.

Probably best if you email me: etmillin@gmail.com

On Thu, Oct 29, 2015 at 8:43 AM, Martin Brousseau notifications@github.com
wrote:

Hi 0x1mason!

I can see the component with GDAL no problem, here is an output I copy
from a test:

[0] "GRIB_COMMENT=Temperature [K]"  string
[1] "GRIB_ELEMENT=TMP"  string
[2] "GRIB_FORECAST_SECONDS=0 sec"   string
[3] "GRIB_PDS_PDTN=0"   string
[4] "GRIB_PDS_TEMPLATE_NUMBERS=0 0 2 0 81 0 0 0 1 0 0 0 0 103 0 0 0 0 2 255 0 0 0 0 0"  string
[5] "GRIB_REF_TIME=  1445860800 sec UTC"    string
[6] "GRIB_SHORT_NAME=2-HTGL"    string
[7] "GRIB_UNIT=[K]" string
[8] "GRIB_VALID_TIME=  1445860800 sec UTC"  string

[0] "GRIB_COMMENT=u-component of wind [m/s]"    string
[1] "GRIB_ELEMENT=UGRD" string
[2] "GRIB_FORECAST_SECONDS=0 sec"   string
[3] "GRIB_REF_TIME=  1445860800 sec UTC"    string
[4] "GRIB_SHORT_NAME=10-HTGL"   string
[5] "GRIB_UNIT=[m/s]"   string
[6] "GRIB_VALID_TIME=  1445860800 sec UTC"  string

[0] "GRIB_COMMENT=v-component of wind [m/s]"    string
[1] "GRIB_ELEMENT=VGRD" string
[2] "GRIB_FORECAST_SECONDS=0 sec"   string
[3] "GRIB_REF_TIME=  1445860800 sec UTC"    string
[4] "GRIB_SHORT_NAME=10-HTGL"   string
[5] "GRIB_UNIT=[m/s]"   string
[6] "GRIB_VALID_TIME=  1445860800 sec UTC"  string

[0] "GRIB_COMMENT=03 hr Total precipitation [kg/(m^2)]" string
[1] "GRIB_ELEMENT=APCP03"   string
[2] "GRIB_FORECAST_SECONDS=356400 sec"  string
[3] "GRIB_REF_TIME=  1445860800 sec UTC"    string
[4] "GRIB_SHORT_NAME=0-SFC" string
[5] "GRIB_UNIT=[kg/(m^2)]"  string
[6] "GRIB_VALID_TIME=  1446228000 sec UTC"  string

[0] "GRIB_COMMENT=Temperature [K]"  string
[1] "GRIB_ELEMENT=TMP"  string
[2] "GRIB_FORECAST_SECONDS=367200 sec"  string
[3] "GRIB_REF_TIME=  1445860800 sec UTC"    string
[4] "GRIB_SHORT_NAME=2-HTGL"    string
[5] "GRIB_UNIT=[K]" string
[6] "GRIB_VALID_TIME=  1446228000 sec UTC"  string

—
Reply to this email directly or view it on GitHub
#13 (comment)
.

0x1mason · 2015-10-29T17:36:29Z

@silv2015

I have one more guess: the problem could be your v-comp messages use multiple fields, which isn't enabled by default. Try this:

using (GribFile file = new GribFile(somePath))
{
  file.Context.EnableMultipleFieldMessages = true;
  // do stuff
}

I haven't test this feature yet. Will do once I get appropriate test data. I also may have found some data at the CISL site, but trying to find some that aren't 200+ GB.

Also, I just noticed there's a bug in the getter for EnableMultipleFieldMessages, but that shouldn't make any difference for setting it.

0x1mason · 2015-10-29T17:44:06Z

Similar issue here: https://code.zmaw.de/boards/1/topics/2835

silv2015 · 2015-10-30T12:11:51Z

Good Morning 0x1mason!

I just arrived at the office, thanks for the suggestion!

I will try it today and if I still cannot load it, I will ask how we can arrange so that you can have the file I'm using

silv2015 · 2015-10-30T13:05:55Z

YES!!! This was it!

I added file.Context.EnableMultipleFieldMessages = true;

and now I can see the v component... Man I need to review the info I have on the Grib 2 format!

Thanks a lot!!!

0x1mason · 2015-10-30T13:18:41Z

Woot! Glad it worked.
On Oct 30, 2015 9:05 AM, "Martin Brousseau" notifications@github.com
wrote:

YES!!! This was it!

I added file.Context.EnableMultipleFieldMessages = true;

and now I can see the v component... Man I need to review the info I have
on the Grib 2 format!

Thanks a lot!!!

—
Reply to this email directly or view it on GitHub
#13 (comment)
.

0x1mason · 2015-10-30T14:17:37Z

Will add note to Usage section about this.

Thanks again for trying the library. Let me know if you have further issues.

silv2015 changed the title ~~Missing channel with Grib 2 files~~ Missing Element with Grib 2 files Oct 27, 2015

0x1mason added the bug label Oct 28, 2015

0x1mason closed this as completed Oct 30, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Missing Element with Grib 2 files #13

Missing Element with Grib 2 files #13

silv2015 commented Oct 27, 2015

0x1mason commented Oct 28, 2015

0x1mason commented Oct 28, 2015

0x1mason commented Oct 28, 2015

silv2015 commented Oct 28, 2015

silv2015 commented Oct 28, 2015

0x1mason commented Oct 28, 2015

silv2015 commented Oct 29, 2015

0x1mason commented Oct 29, 2015

0x1mason commented Oct 29, 2015

0x1mason commented Oct 29, 2015

silv2015 commented Oct 30, 2015

silv2015 commented Oct 30, 2015

0x1mason commented Oct 30, 2015

0x1mason commented Oct 30, 2015

Missing Element with Grib 2 files #13

Missing Element with Grib 2 files #13

Comments

silv2015 commented Oct 27, 2015

0x1mason commented Oct 28, 2015

0x1mason commented Oct 28, 2015

0x1mason commented Oct 28, 2015

silv2015 commented Oct 28, 2015

silv2015 commented Oct 28, 2015

0x1mason commented Oct 28, 2015

silv2015 commented Oct 29, 2015

0x1mason commented Oct 29, 2015

0x1mason commented Oct 29, 2015

0x1mason commented Oct 29, 2015

silv2015 commented Oct 30, 2015

silv2015 commented Oct 30, 2015

0x1mason commented Oct 30, 2015

0x1mason commented Oct 30, 2015