Testing Report for Raster Data Source in Cloud Storage

Posted by Xiaoyan(Sharon) Mu on 2018-08-15
Estimated Reading Time 4 Minutes
Words 761 In Total
Viewed Times

For the purpose of assisting and sharing , I will try to write in English.

The new tool Create Cloud Storage Connection File is first introduced to ArcGIS Desktop 10.6.1 which is the latest version until now. The [Create Cloud Storage Connection File](file:///C:/Program%20Files/ArcGIS/Pro/Resources/Help/pro-app/tool-reference/data-management/create-cloud-storage-connection-file.htm) tool creates a connection file (*.acs) for ArcGIS-supported cloud storage. It allows existing raster geoprocessing tools to write cloud raster format (CRF) datasets into the cloud storage bucket or read raster datasets (not limited to CRF) stored in the cloud storage as input.

This is an amazing new feature of ArcGIS Desktop, including ArcMap 10.6.1 and ArcGIS Pro 2.2. The stuff from one of our partners and I are doing some researching on the new capability. This article is something about the testing report, which indicates the feasibility of various workflows for adding raster datasets to mosaic dataset.


Local Testing Environment

  • Latest Esri Technologies

    • ArcMap 10.6.1

    • ArcGIS Pro 2.2.1

  • Windows 10 Pro 64 bit English Version



Huawei Cloud

ArcMap

  • Steps

    1. Create Huawei Cloud connection file (√)
    2. Add raster to a local mosaic dataset (√)
  • Conclusion

    • Bingo ! This workflow for source raster dataset in Huawei Cloud is perfect.

ArcGIS Pro

  • Steps

    1. Create Huawei Cloud connection file (√)

    2. Add Raster to Mosaic Dataset (×)

      • Error Message as below:

        Failed to execute. Parameters are not valid.

        Error: 000732 Input Data: Dataset …\HuaWeiConnection.acs\…LGN00_MTL.txt does not exist or is not supported

    3. Trying workaround: Add Raster to Mosaic Dataset using arcpy (python 3.x for ArcGIS Pro).

      For the purpose of verification , Add Raster to Mosaic Dataset using arcpy (python 2.x for ArcMap), it works. (√)

  • Conclusion

    • It should be a bug of ArcGIS Pro, that the tool Add Rasters To Mosaic Dataset can not recognize the logical file path including cloud connection. The tool will justify the path if it exists. The path containing cloud connection is a logical path , not a physical path.  The same input file path works in ArcMap.

    • Internal case submitted #02166069 - closed

      Related bug: BUG-000116191

      Workaround: Store the rasters in the root of the bucket, and add them from that location.


ALiBaBa Cloud

In China, we also call it ALiYun.

ArcMap

  • Steps

    1. Create ALiBaBa Cloud connection file (×)

      The connection file was created, but there was a warning: WARNING 002574: Could not connect to cloud storage in the log

    2. Create a mosaic dataset in a local file geodatabase (√)

    3. Add raster to mosaic dataset (×)

      An error occurred: Error: 8004205f: No new mosaic dataset item was added.

      This error may due to the warning before, I think.

  • Conclusion

    • It may be a bug related to creating connection file to ALiBaBa Cloud Storage. ArcPy (python 27) for ArcMap didn’t work, either.

    • Extra diagnose

      • Using Fiddler to capture the HTTP(S) requests…

        I have used Fiddler Software to catch the HTTP(S) requests made by ArcMap, but nothing was found. I had enable the https capture before testing. At the same time , I checked the Windows Task Manager and Resource Monitor that there was a lot of network activities.

      • Using Wireshark to capture the network transaction …

        Bad TCP records were found.

ArcGIS Pro

  • Steps

    1. Create ALiBaBa Cloud connection file (×)
    2. Create a mosaic dataset in a local file geodatabase (√)
    3. Add raster to mosaic dataset (×)
  • Conclusion

    • Same issue with using HUAWEI cloud source data, and the issue points to the tool not to the cloud itself.
    • ArcPy (python 36) for ArcGIS Pro didn’t work, either.

Internal case submitted #02157762

Related bug: BUG-000116297

Workaround: Internal Version 9226


Custom Cloud

using s3 compatible cloud, for example, Tencent Cloud.


Other Issue

failure with file Geodatabase transfer



The information above is valid as the day it published, I will try my best to have it up to date.

Thank you for reviewing.


如果您喜欢此博客或发现它对您有用,则欢迎对此发表评论。 也欢迎您共享此博客,以便更多人可以参与。 如果博客中使用的图像侵犯了您的版权,请与作者联系以将其删除。 谢谢 !