Socrata 0.0.3-beta

tyler-socrata-net-sdk

.NET Socrata SDK for reading and writing to Socrata APIs

SocrataClient

// Create the client
ISocrataClient client = new SocrataClient(new Uri("https://{domain}.data.socrata.com"), "apikey", "apisecret");

// Optionally validate the client connection
bool IsValidConnection = client.ValidateConnection();

Socrata Resources

// Get a resource by its ID*
Resource dataset = client.GetResource("abcd-1234");

Note Dataset IDs

Working with Resources

Data Manipulation

Socrata Data Types and You

Notes:

• The escape character for text values is \
• Dates must be passed in as string
• Datetimes will not be converted to specific timezones, so be sure all the dates are in the correct/same timezone when they are exported
• Decimals are demarcated with a .
• All of these will be converted to JSON when sent to Socrata, so when in doubt, consult the

Geospatial Data

While the Socrata platform does support geocoding, it is not supported via this SDK currently. Any geospatial data will need to converted into Well-Known Text (WKT) with WGS84 (ESPG 4326) Latitude and Longitude values.

In general, a decimal precision of 5 is sufficient.

Direct Row Manipulation

Insert

// Insert Functionality
List<Dictionary<string, string>> newRecords = new List<Dictionary<string, string>> 
{
  new Dictionary<string, string>{{"api_field", "nextvalue"},{"other_field", "test"}}
};
Result result = dataset.Rows().Insert(newRecords);

// Check if it was successful
Assert.IsFalse(Result.IsError);
// Check the results
Assert.AreEqual(newRecords.Length, result.Inserted);

Update

// Update Functionality
List<Dictionary<string, object>> recordsToUpdate = new List<Dictionary<string, object>> 
{
  new Dictionary<string, object>{{"row_identifier", 1},{"column_that_changed", "newvalue"}}
};
Result result = dataset.Rows().Update(recordsToUpdate);

// Check if it was successful
Assert.IsFalse(Result.IsError);
// Check the results
Assert.AreEqual(recordsToUpdate.Length, result.Updated);

Delete

// Delete Functionality
List<Dictionary<string, string>> recordsToDelete = new List<Dictionary<string, string>> 
{
  new Dictionary<string, string>{{"row_identifier", "ID-1"}}
};
Result result = dataset.Rows().Delete(recordsToDelete);

// Check if it was successful
Assert.IsFalse(Result.IsError);
// Check the results
Assert.AreEqual(recordsToDelete.Length, result.Deleted);

Dataset Management API

Full Documentation

Revision Types

Replace: A full replace of the dataset Update: A upsert of the contents based off the row identifier Delete: A delete of the contents based off the row identifier

File Types

Using an Import Config

Small (in memory) file uploads

  Resource resource = socrataClient.GetResource("tzmz-8bnb");
  Revision revision = resource.OpenRevision(RevisionType.REPLACE);
  Source source = revision.CreateUploadSource("test-csv.csv");
  string filepath = @"C:\Path\To\MyFile.csv";
  string csv = System.IO.File.ReadAllText(filepath);
  source.AddBytesToSource(csv);
  source.AwaitCompletion(status => Console.WriteLine(status));
  revision.Apply();
  revision.AwaitCompletion(status => Console.WriteLine(status));

Large (Streaming) file uploads

  Resource resource = socrataClient.GetResource("tzmz-8bnb");
  Revision revision = resource.OpenRevision(RevisionType.REPLACE);
  Source source = revision.CreateStreamingSource("test-csv.csv");
  ByteSink sink = source.StreamingSource(ContentType.CSV);
  //Open the stream and read the file.
  sink.FileSink(@"C:\Path\To\MyFile.csv");
  source.AwaitCompletion(status => Console.WriteLine(status));
  revision.Apply();
  revision.AwaitCompletion(status => Console.WriteLine(status));

Handling error rows

  if(source.HasErrorRows())
  {
    string errorFile = "C:\Path\To\ErrorRows.csv";
    int numberOfErrors = source.NumberOfErrors();
    source.ExportErrorRows(errorFile);
    // Throw an exception
    throw new Exception($"{numberOfErrors} Error(s) were found in the upload");
  }

Dataset Metadata Only Revisions

  Resource resource = socrataClient.GetResource("tzmz-8bnb");
  Revision revision = resource.OpenRevision(RevisionType.REPLACE);
  revision.CreateViewSource();
  revision.SetDescription("New description for the <b>resource</b>");
  revision.RenameResource("New Name");
  revision.Apply();
  revision.AwaitCompletion(status => Console.WriteLine(status));

Column Metadata Revisions

  Resource resource = socrataClient.GetResource("tzmz-8bnb");
  Revision revision = resource.OpenRevision(RevisionType.REPLACE);
  Source source = revision.CreateViewSource();
  revision.Apply();

Reading Data

You can also use the Rows() to access records in the dataset

/*
Example custom class
[DataContract]
class MyResource
{
  [DataMember(Name = "column_field")]
  public string ColumnField { get; internal set; }
}
*/
Resource resource = socrataClient.GetResource("tzmz-8bnb");
Rows rows = resource.Rows();
List<MyResource> allRecords = rows.FetchAll<MyResource>();
long limit = 1000;
long offset = 0;
List<MyResource> someRecords = rows.Fetch<MyResource>(limit, offset);

// Conversely, you can just use a Map<string, object>
List<Map<string, object>> allRecords = rows.FetchAll<Map<string, object>>();

Pagination

You might have a really big dataset, which will require paginating over the result set.

Resource resource = socrataClient.GetResource("tzmz-8bnb");
Rows rows = resource.Rows();
// Get the total number of records
long total = rows.Count();
long limit = 1000;
long offset = 0;
while(offset < total)
{
  List<Map<string, object>> allRecords = rows.Fetch<Map<string, object>>(limit, offset);
  offset += limit;
}

Metadata

Accessing Resource Metadata

Resource dataset = client.GetResource("abcd-1234");
string resourceName = dataset.metadata.Name;

Running Tests

dotnet test

Or a specific test:

dotnet test --filter {testname}