CoastSat_WRL/example_jupyter.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# *CoastSat*: example at Narrabeen-Collaroy, Australia\n",
    "\n",
    "This software is described in *Vos K., Splinter K.D., Harley M.D., Simmons J.A., Turner I.L. (submitted). CoastSat: a Google Earth Engine-enabled software to extract shorelines from publicly available satellite imagery, Environmental Modelling and Software*. It enables the users to extract time-series of shoreline change over the last 30+ years at their site of interest.\n",
    "\n",
    "There are two main steps:\n",
    "- retrieval of the satellite images of the region of interest from Google Earth Engine\n",
    "- extraction of the shorelines from the images using a sub-pixel resolution technique\n",
    "\n",
    "## 1. Initial settings\n",
    "\n",
    "Refer to the **Installation** section of the README for instructions on how to install the Python packages necessary to run the software, including Google Earth Engine Python API. If that step has been completed correctly, the following packages should be imported without any problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import pickle\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "import matplotlib.pyplot as plt\n",
    "import SDS_download, SDS_preprocess, SDS_shoreline, SDS_tools, SDS_transects"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Retrieval of the images from GEE\n",
    "\n",
    "Define the region of interest (`polygon`), the date range (`dates`) and the satellite missions (`sat_list`) from which you wish to retrieve the satellite images. The images will be cropped on the Google Earth Engine server and only the region of interest will be downloaded as a .TIF file. The files will be organised in the local directory under *.\\data\\sitename*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# region of interest (longitude, latitude)\n",
    "polygon = [[[151.2957545, -33.7012561],\n",
    "            [151.297557, -33.7388075],\n",
    "            [151.312234, -33.7390216],\n",
    "            [151.311204, -33.701399],\n",
    "            [151.2957545, -33.7012561]]]   \n",
    "# date range\n",
    "dates = ['2017-12-01', '2018-01-01']\n",
    "# satellite missions\n",
    "sat_list = ['S2']\n",
    "# name of the site\n",
    "sitename = 'NARRA'\n",
    "# put all the inputs into a dictionnary\n",
    "inputs = {'polygon': polygon, 'dates': dates, 'sat_list': sat_list, 'sitename': sitename}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Retrieve satellite images from GEE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metadata = SDS_download.retrieve_images(inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**If you have already retrieved the images**, just load the metadata file by only running the section below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filepath = os.path.join(os.getcwd(), 'data', sitename)\n",
    "with open(os.path.join(filepath, sitename + '_metadata' + '.pkl'), 'rb') as f:\n",
    "    metadata = pickle.load(f) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Shoreline extraction\n",
    "\n",
    "Maps the position of the shoreline on the satellite images. The user can define the cloud threhold (`cloud_thresh`) and select the spatial reference system in which he would like to output the coordinates of the mapped shorelines (`output_epsg`). See http://spatialreference.org/ to find the EPSG number corresponding to your local coordinate system. To quality control each shoreline detection and manually validate the mapped shorelines, the user has the option to set the parameter `check_detection` to **True**. The other parameters are for advanced users only and are described in the last section of the README."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "settings = { \n",
    "    # general parameters:\n",
    "    'cloud_thresh': 0.2,        # threshold on maximum cloud cover\n",
    "    'output_epsg': 28356,       # epsg code of spatial reference system desired for the output   \n",
    "    # quality control:\n",
    "    'check_detection': True,    # if True, shows each shoreline detection to the user for validation\n",
    "\n",
    "    # add the inputs defined previously\n",
    "    'inputs': inputs,\n",
    "    \n",
    "    # [ONLY FOR ADVANCED USERS] shoreline detection parameters:\n",
    "    'min_beach_area': 4500,     # minimum area (in metres^2) for an object to be labelled as a beach\n",
    "    'buffer_size': 150,         # radius (in metres) of the buffer around sandy pixels considered in the shoreline detection\n",
    "    'min_length_sl': 200,       # minimum length (in metres) of shoreline perimeter to be valid\n",
    "    'cloud_mask_issue': False,  # switch this parameter to True if sand pixels are masked (in black) on many images \n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [OPTIONAL] Save .jpg of the satellite images \n",
    "Saves .jpg files of the preprocessed satellite images (cloud masking + pansharpening/down-sampling) under *./data/sitename/jpeg_files\\preprocessed*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SDS_preprocess.save_jpg(metadata, settings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [OPTIONAL] Digitize a reference shoreline\n",
    "Creates a reference shoreline which helps to identify outliers and false detections. The reference shoreline is manually digitised by the user on one of the images. The parameter `max_dist_ref` defines the maximum distance from the reference shoreline (in metres) at which a valid detected shoreline can be. If you think that you shoreline will move more than the default value of 100 m, please change this parameter to an appropriate distance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib qt\n",
    "settings['reference_shoreline'] = SDS_preprocess.get_reference_sl_manual(metadata, settings)\n",
    "settings['max_dist_ref'] = 100 # max distance (in meters) allowed from the reference shoreline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Batch shoreline detection\n",
    "Extracts the shorelines from the images. The mapped shorelines are saved into `output.pkl` (under *./data/sitename*) which contains the shoreline coordinates for each date in the spatial reference system specified by the user in `'output_epsg'`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%matplotlib qt\n",
    "output = SDS_shoreline.extract_shorelines(metadata, settings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Simple plot of the mapped shorelines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure()\n",
    "plt.axis('equal')\n",
    "plt.xlabel('Eastings')\n",
    "plt.ylabel('Northings')\n",
    "plt.grid(linestyle=':', color='0.5')\n",
    "for i in range(len(output['shorelines'])):\n",
    "    sl = output['shorelines'][i]\n",
    "    date = output['dates'][i]\n",
    "    plt.plot(sl[:,0], sl[:,1], '.', label=date.strftime('%d-%m-%Y'))\n",
    "plt.legend()\n",
    "mng = plt.get_current_fig_manager()                                         \n",
    "mng.window.showMaximized()    \n",
    "fig.set_size_inches([15.76,  8.52])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Shoreline analysis\n",
    "\n",
    "In this section we show how to compute time-series of cross-shore distance along user-defined shore-normal transects."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**If you have already mapped the shorelines**, just load the output file by only running the section below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filepath = os.path.join(os.getcwd(), 'data', sitename)\n",
    "with open(os.path.join(filepath, sitename + '_output' + '.pkl'), 'rb') as f:\n",
    "    output = pickle.load(f) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The shore-normal transects are defined by two points, the origin of the transect and a second point that defines its orientaion. The parameter *transect_length* determines how far from the origin the transects span."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "settings['transect_length'] = 500 # defines the length of the transects in metres"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 3 options to define the coordinates of the shore-normal transects:\n",
    "\n",
    "**Option 1**: the user can interactively draw the shore-normal transects along the beach by calling:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib qt\n",
    "transects = SDS_transects.draw_transects(output, settings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Option 2**: the user can load the transect coordinates (make sure the spatial reference system is the same as defined by *output_epsg* previously) from a .kml file by calling:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "kml_file = 'NARRA_transects.kml'\n",
    "transects = SDS_transects.load_transects_from_kml(kml_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Option 3**: manually provide the coordinates of the transects as shown in the example below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transects = dict([])\n",
    "transects['Transect 1'] = np.array([[342836, 6269215], [343315, 6269071]])\n",
    "transects['Transect 2'] = np.array([[342482, 6268466], [342958, 6268310]])\n",
    "transects['Transect 3'] = np.array([[342185, 6267650], [342685, 6267641]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, intersect the transects with the 2D shorelines to obtain time-series of cross-shore distance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# defines the along-shore distance over which to consider shoreline points to compute the median intersection (robust to outliers)\n",
    "settings['along_dist'] = 25 \n",
    "cross_distance = SDS_transects.compute_intersection(output, transects, settings) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the time-series of shoreline change along each transect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import gridspec\n",
    "import numpy as np\n",
    "fig = plt.figure()\n",
    "gs = gridspec.GridSpec(len(cross_distance),1)\n",
    "gs.update(left=0.05, right=0.95, bottom=0.05, top=0.95, hspace=0.05)\n",
    "for i,key in enumerate(cross_distance.keys()):\n",
    "    ax = fig.add_subplot(gs[i,0])\n",
    "    ax.grid(linestyle=':', color='0.5')\n",
    "    ax.set_ylim([-50,50])\n",
    "    if not i == len(cross_distance.keys()):\n",
    "        ax.set_xticks = []\n",
    "    ax.plot(output['dates'], cross_distance[key]- np.nanmedian(cross_distance[key]), '-^', markersize=6)\n",
    "    ax.set_ylabel('distance [m]', fontsize=12)\n",
    "    ax.text(0.5,0.95,'Transect ' + key, bbox=dict(boxstyle=\"square\", ec='k',fc='w'), ha='center',\n",
    "            va='top', transform=ax.transAxes, fontsize=14)\n",
    "mng = plt.get_current_fig_manager()                                         \n",
    "mng.window.showMaximized()    \n",
    "fig.set_size_inches([15.76,  8.52])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}