New Jupyter Notebook for CARRA on Height Levels - Downloading datasets and Preparing them for analysis #180
Comments
|
Hi @FabioMangini, The notebook you shared is empty. |
|
Hi @malmans2, Here's a new notebook: I created it to download:
between September 1990 (in the text above I mistakenly referred to January 1992) and November 2024 . I have an issue with this script. I get the following the error message when I run it:
I was wondering whether you could help me solve this issue. |
|
Hi @FabioMangini, Is the notebook intended to download all the data you need? If you'd like me to cache the data, I can do that and will notify you once it's available on the VM. Regarding the issue: In the old VM there used to be a shared Create your own config file in import os
os.environ["CDSAPI_RC"] = os.path.expanduser("~/mangini_fabio/.cdsapirc")If you don't know how to create your .cdsapirc, you can follow these instructions: https://github.com/ecmwf/cdsapi?tab=readme-ov-file#configure |
|
Hi @malmans2, Thanks for the explanation. I will follow the procedure you suggest.
Yes, it would be great if you cached the data. Thanks! I would like to ask you two more questions about the Jupyter Notebook:
|
This can be tricky as we need to access the whole timeseries at once (i.e., we have to use
This shouldn't be a problem as the data is chunked over time. Using Let's resume the discussion once the data is on disk. I have a few meetings now. I'll have a look to the code you shared in the afternoon and I will start downloading data if I have no questions. |
|
Hi @malmans2, perfect! Thank you very much. |
|
Hi @FabioMangini, I still didn't get to cache your data. I'm doing the same for other evaluators and it's taking longer than what I thought. |
|
Hi @malmans2, It is fine with me. Thank you very much for letting me know and for your help. |
|
Hi @FabioMangini, I started downloading the data over the weekend, but it will take some time. We’ll likely need at least a couple more days. I’ll let you know once it’s done. By the way, the notebook you shared has the wrong |
|
Hi @FabioMangini, I finished downloading CARRA East. However, CARRA West is taking a very long time. I'm afraid it won't be ready before next week. If you want to start playing with the timeseries of CARRA East, I've implemented and cached your second requirement in this comment: #180 (comment) Next, I will look into computing the climatological map. Please do not try to use the template with CARRA West as it's not ready. |
|
Hi @malmans2, I see! Thank you very much. |
|
Hi @FabioMangini, I'm still downloading CARRA West Domain. The dateset is huge and probably it's archived on tape. |
|
Hi @malmans2, Ok! Thank you for letting me know. |
|
Hi @FabioMangini, CARRA West is ready. I made a few adjustments to the code, so please use the latest template: https://github.com/bopen/c3s-eqc-toolbox-template/blob/main/notebooks/wp5/carra_wind_power_estimation.ipynb I'm now downloading ERA5. |
|
@FabioMangini is this data available in csv format? It would be much easier to handle. Otherwise, I'll need to install another dependency on the VM. |
Hi @malmans2! Thank you very much for the notebook. |
Unfortunately, I cannot find the data in csv format. I will keep searching and I will let you later today. |
|
OK, in the meantime I'm caching both ERA5 and the climatology maps. |
|
Hi @FabioMangini, The climatology maps are ready. As usual, I've added a couple o quick and dirty plots just to get you started. I'll let you know when ERA5 is ready. Here is the latest template: https://github.com/bopen/c3s-eqc-toolbox-template/blob/main/notebooks/wp5/carra_wind_power_estimation.ipynb |
|
Hi @malmans2!
Thanks!
I checked it, but it seems that the data are not provided in csv format. |
|
OK, I'll install the new dependency overnight and will try to read it tomorrow. |
|
@FabioMangini there are long queues in the CDS right now, so I'm still downloading ERA5. |
|
Hi @malmans2, Ok! Thank you very much! |
|
Hi @FabioMangini, All the features you requested should be implemented now. See: https://github.com/bopen/c3s-eqc-toolbox-template/blob/main/notebooks/wp5/carra_wind_power_estimation.ipynb Let me know if you have everything you need to develop your notebook. |
|
Hi @malmans2, Ok! Thank you very much. |
Notebook description
Good morning,
I would like to create a Jupyter Notebook that evaluates the ability of ERA5 and CARRA to estimate the energy production of onshore wind farms in northern Norway. I am writing to you because I was wondering if you could help me with downloading the datasets and preparing them for the analysis.
The energy produced by a wind turbine is related to the wind speed at the turbine's height. So, I was wondering whether you could create a Jupyter Notebook that downloads the:
zonal and meridional components of the wind field from the ERA5 hourly reanalysis data on single levels at 100m height (https://cds.climate.copernicus.eu/datasets/reanalysis-era5-single-levels?tab=download)
wind speed from CARRA-West and CARRA-East on height levels at 75m and 100m height (https://cds.climate.copernicus.eu/datasets/reanalysis-carra-height-levels?tab=download)
since January 1992.
Once the datasets have been downloaded, I was wondering whether you compute the climatological mean of the wind speed from CARRA-West and CARRA-East? They would help identify the locations that are most suitable for setting up new wind farms by showing climatological wind speed values.
The Jupyter Notebook would also like to focus on individual wind farms and understand how well ERA5 and CARRA can estimate their energy production. For simplicity, the Jupyter Notebook would first consider a single wind farm. This is the Dønnesfjord wind farm, which is located at 70.63°N and 22.46°E and became operative in 2022. So, I was wondering if you could create a script that extracts the wind speed from ERA5 and CARRA-East at 70.63°N and 22.46°E since January 2022.
To conclude, I was also wondering if it were possible for you to use python to download and store into a pandas DataFrame the wind power produced by the Norwegian wind farm. The data are available at: https://www.nve.no/media/16891/vindprod2002-2023_kraftverk_utcplus1.xlsx.
Thank you very much.
Notebook link or upload
wind_power_estimation.ipynb.zip
Anything else we need to know?
No response
Environment
name: wp5
channels:
dependencies:
prefix: /data/common/miniforge3/envs/wp5
The text was updated successfully, but these errors were encountered: