AR1/docs/import/esri.md

# ESRI/ArcGIS Import

Import spatial data from ArcGIS Server and ArcGIS Online services.

## Overview

ESRI/ArcGIS import allows you to harvest and import data from ArcGIS REST services, including MapServer and FeatureServer endpoints.

## Supported Services

### MapServer

- **Description**: ArcGIS MapServer REST service
- **Features**: Map layers, feature layers, group layers
- **Use Case**: Static map services, published datasets

### FeatureServer

- **Description**: ArcGIS FeatureServer REST service
- **Features**: Editable feature layers, queries
- **Use Case**: Dynamic data, editable services

### ArcGIS Online

- **Description**: ArcGIS Online hosted services
- **Features**: Public and private services
- **Use Case**: Cloud-hosted datasets

## Import Methods

### Service Browser

Browse and import from ArcGIS services:

1. Navigate to ESRI browser page
2. Enter service URL
3. Browse available layers
4. Select layer to import
5. Configure import options
6. Click "Import"

### Direct URL Import

Import directly from service URL:

1. Navigate to URL import page
2. Enter ArcGIS service URL
3. System detects service type
4. Configure import options
5. Click "Import"

## Service URL Format

### MapServer

```
https://server/arcgis/rest/services/ServiceName/MapServer
https://server/arcgis/rest/services/ServiceName/MapServer/{layerId}
```

### FeatureServer

```
https://server/arcgis/rest/services/ServiceName/FeatureServer
https://server/arcgis/rest/services/ServiceName/FeatureServer/{layerId}
```

### ArcGIS Online

```
https://services.arcgis.com/{orgId}/arcgis/rest/services/{serviceName}/FeatureServer
```

## Import Process

### Step 1: Service Discovery

System discovers service information:
- Service type (MapServer/FeatureServer)
- Available layers
- Layer metadata
- Spatial reference system

### Step 2: Layer Selection

Select layer to import:
- Browse available layers
- View layer metadata
- Check geometry types
- Verify attribute fields

### Step 3: Query Configuration

Configure data query:
- **Where Clause**: SQL WHERE clause for filtering
- **Out Fields**: Fields to include
- **Spatial Filter**: Bounding box or geometry
- **Max Records**: Maximum features to import

### Step 4: Authentication

If required, provide credentials:
- **Username/Password**: ArcGIS credentials
- **Token**: ArcGIS token (auto-generated)
- **Anonymous**: For public services

### Step 5: Data Harvesting

Data is harvested from service:
- Paginated queries (1000 features per batch)
- Geometry conversion (ArcGIS JSON to GeoJSON)
- Attribute extraction
- Coordinate transformation

### Step 6: Import

Harvested data is imported:
- Table creation: `spatial_data_{file_id}`
- Geometry conversion to PostGIS
- Spatial index creation
- Metadata storage

## Query Parameters

### Where Clause

SQL WHERE clause for filtering:

```sql
OBJECTID > 1000
Category = 'Residential'
Population > 50000
```

### Out Fields

Comma-separated list of fields:

```
OBJECTID,Name,Category,Population
*  -- All fields
```

### Spatial Filter

Bounding box or geometry:

```json
{
  "geometry": {
    "xmin": -180,
    "ymin": -90,
    "xmax": 180,
    "ymax": 90,
    "spatialReference": {"wkid": 4326}
  },
  "geometryType": "esriGeometryEnvelope"
}
```

## Authentication

### Public Services

No authentication required for public services.

### Secured Services

For secured services, provide:
- **Username**: ArcGIS username
- **Password**: ArcGIS password
- **Token**: Auto-generated from credentials

### Token Management

- Tokens auto-generated from credentials
- Tokens cached for session
- Token refresh handled automatically

## Scheduled Imports

Set up recurring imports:

1. Configure ArcGIS import
2. Set schedule (daily, weekly, monthly)
3. Configure update mode
4. Save schedule

**Update Modes**:
- **Replace**: Replace all data
- **Append**: Add new data
- **Upsert**: Update existing, insert new

## Metadata Harvesting

System harvests comprehensive metadata:
- Service information
- Layer metadata
- Field definitions
- Spatial reference
- Extent information

## Example: MapServer Import

```bash
# Via browser
1. Navigate to ESRI browser
2. Enter: https://server/arcgis/rest/services/ServiceName/MapServer
3. Select layer
4. Configure query
5. Click "Harvest"
```

## Example: FeatureServer Import

```json
{
  "service_url": "https://server/arcgis/rest/services/ServiceName/FeatureServer/0",
  "where": "Category = 'Residential'",
  "out_fields": "*",
  "max_records": 10000,
  "auth_username": "user",
  "auth_password": "pass"
}
```

## Troubleshooting

### Common Issues

**Service not accessible**
- Verify service URL
- Check network connectivity
- Verify service is public or provide credentials

**Authentication failed**
- Verify username/password
- Check service permissions
- Verify token endpoint accessible

**No features returned**
- Check WHERE clause syntax
- Verify layer has data
- Check spatial filter bounds

**Import timeout**
- Reduce max records
- Use spatial filter to limit data
- Consider scheduled import for large datasets

**Geometry errors**
- Verify spatial reference system
- Check for invalid geometries
- Verify geometry type compatibility

## Related Documentation

- [Vector Import](vector.md)
- [Raster Import](raster.md)
- [PostGIS Import](postgis.md)
- [URL Import](vector.md#url-import)