Testing Report for Raster Data Source in Cloud Storage

For the purpose of assisting and sharing , I will try to write in English.

The new tool Create Cloud Storage Connection File is first introduced to ArcGIS Desktop 10.6.1 which is the latest version until now. The [Create Cloud Storage Connection File](file:///C:/Program%20Files/ArcGIS/Pro/Resources/Help/pro-app/tool-reference/data-management/create-cloud-storage-connection-file.htm) tool creates a connection file (*.acs) for ArcGIS-supported cloud storage. It allows existing raster geoprocessing tools to write cloud raster format (CRF) datasets into the cloud storage bucket or read raster datasets (not limited to CRF) stored in the cloud storage as input.

This is an amazing new feature of ArcGIS Desktop, including ArcMap 10.6.1 and ArcGIS Pro 2.2. The stuff from one of our partners and I are doing some researching on the new capability. This article is something about the testing report, which indicates the feasibility of various workflows for adding raster datasets to mosaic dataset.

Local Testing Environment

• Latest Esri Technologies

  • ArcMap 10.6.1

  • ArcGIS Pro 2.2.1

• Windows 10 Pro 64 bit English Version

Huawei Cloud

ArcMap

• Steps

  1. Create Huawei Cloud connection file (√)
  2. Add raster to a local mosaic dataset (√)

• Conclusion

  • Bingo ! This workflow for source raster dataset in Huawei Cloud is perfect.

ArcGIS Pro

• Steps

  1. Create Huawei Cloud connection file (√)

  2. Add Raster to Mosaic Dataset (×)

     • Error Message as below:

       Failed to execute. Parameters are not valid.

       Error: 000732 Input Data: Dataset …\HuaWeiConnection.acs\…LGN00_MTL.txt does not exist or is not supported

  3. Trying workaround: Add Raster to Mosaic Dataset using arcpy (python 3.x for ArcGIS Pro).

     For the purpose of verification , Add Raster to Mosaic Dataset using arcpy (python 2.x for ArcMap), it works. (√)

• Conclusion

  • It should be a bug of ArcGIS Pro, that the tool Add Rasters To Mosaic Dataset can not recognize the logical file path including cloud connection. The tool will justify the path if it exists. The path containing cloud connection is a logical path , not a physical path.  The same input file path works in ArcMap.

  • Internal case submitted #02166069 - closed

    Related bug: BUG-000116191

    Workaround: Store the rasters in the root of the bucket, and add them from that location.

  
  

ALiBaBa Cloud

In China, we also call it ALiYun.

ArcMap

• Steps

  1. Create ALiBaBa Cloud connection file (×)

     The connection file was created, but there was a warning: WARNING 002574: Could not connect to cloud storage in the log

  2. Create a mosaic dataset in a local file geodatabase (√)

  3. Add raster to mosaic dataset (×)

     An error occurred: Error: 8004205f: No new mosaic dataset item was added.

     This error may due to the warning before, I think.

• Conclusion

  • It may be a bug related to creating connection file to ALiBaBa Cloud Storage. ArcPy (python 27) for ArcMap didn’t work, either.

  • Extra diagnose

    • Using Fiddler to capture the HTTP(S) requests…

      I have used Fiddler Software to catch the HTTP(S) requests made by ArcMap, but nothing was found. I had enable the https capture before testing. At the same time , I checked the Windows Task Manager and Resource Monitor that there was a lot of network activities.

    • Using Wireshark to capture the network transaction …

      Bad TCP records were found.

ArcGIS Pro

• Steps

  1. Create ALiBaBa Cloud connection file (×)
  2. Create a mosaic dataset in a local file geodatabase (√)
  3. Add raster to mosaic dataset (×)

• Conclusion

  • Same issue with using HUAWEI cloud source data, and the issue points to the tool not to the cloud itself.
  • ArcPy (python 36) for ArcGIS Pro didn’t work, either.

Internal case submitted #02157762

Related bug: BUG-000116297

Workaround: Internal Version 9226

Custom Cloud

using s3 compatible cloud, for example, Tencent Cloud.

• Steps

  1. Create Tencent Cloud connection file using ArcPy (python 27) for ArcMap
  2. Capture network interaction information using WireShark
     • As Amazon Provider selected, the region and endpoint will request Amazon CNAME address. In this condition, I don’t think the custom cloud works for now.

• Internal case submitted

  #02157772 Can Service End Point parameter be customized by users when connecting to the cloud storage ?

Other Issue

failure with file Geodatabase transfer

• Details

  When the file geodatabase was copied to a new location ,which was some scheme of path as before, on the other new machine . The mosaic dataset didn’t show up the source raster data at all.

• Internal case submitted

  #02165287 how to transfer mosaic dataset referencing raster dataset in cloud storage

The information above is valid as the day it published, I will try my best to have it up to date.

Thank you for reviewing.

---

• ← Previous Post
• Next Post →

---

---