I have a database table which currently holds geometric data in SRID 27700 (British National Grid). While retrieving the data however I need to transform it to SRID 4326 (WGS84). Is there any way to apply a function such as ST_Transform found in PostGIS to my data in order to get the result I need?
NOTE: The solution needs to be able to be implemented using T-SQL and not stored procedures etc. I have to be able to construct a statement and have it saved in a table as a string field for retrieval later. This is because my solution is database agnostic.
The way I am currently doing this in Oracle is as follows:
select CLUSTER_ID,
NUM_POINTS,
FEATURE_PK,
A.CELL_CENTROID.SDO_POINT.X,
A.CELL_CENTROID.SDO_POINT.Y,
A.CLUSTER_CENTROID.SDO_POINT.X,
A.CLUSTER_CENTROID.SDO_POINT.Y,
TO_CHAR (A.CLUSTER_EXTENT.GET_WKT ()),
TO_CHAR (A.CELL_GEOM.GET_WKT ()),
A.CLUSTER_EXTENT.SDO_SRID
from (SELECT CLUSTER_ID,
NUM_POINTS,
FEATURE_PK,
SDO_CS.transform (CLUSTER_CENTROID, 4326) cluster_centroid,
CLUSTER_EXTENT,
SDO_CS.transform (CELL_CENTROID, 4326) cell_centroid,
CELL_GEOM FROM :0) a
where sdo_filter( A.CELL_GEOM,
SDO_CS.transform(mdsys.sdo_geometry(2003, :1, NULL, mdsys.sdo_elem_info_array(1,1003,3),mdsys.sdo_ordinate_array(:2, :3, :4, :5)),81989)) = 'TRUE'
In PostgreSQL using PostGIS I am doing it like this:
select CLUSTER_ID,
NUM_POINTS,
FEATURE_PK, ST_X(a.CELL_CENTROID),
ST_Y(a.CELL_CENTROID),
ST_X(ST_TRANSFORM(a.CLUSTER_CENTROID, 4326)),
ST_Y(ST_TRANSFORM(a.CLUSTER_CENTROID, 4326)),
ST_AsText(a.CLUSTER_EXTENT),
ST_AsText(a.CELL_GEOM),
ST_SRID(a.CLUSTER_EXTENT)
FROM (SELECT CLUSTER_ID,
NUM_POINTS,
FEATURE_PK,
ST_TRANSFORM(ST_SetSRID(CLUSTER_CENTROID, 27700), 4326) cluster_centroid,
CLUSTER_EXTENT,
ST_TRANSFORM(ST_SetSRID(CELL_CENTROID, 27700), 4326) cell_centroid,
CELL_GEOM
from :0) AS a
where ST_Intersects(ST_Transform(ST_SetSRID(a.CELL_GEOM, 27700), :1), ST_Transform(ST_GeomFromText('POLYGON(('||:2||' '||:3||', '||:4||' '||:3||', '||:4||' '||:5||', '||:2||' '||:5||', '||:2||' '||:3||'))', 4326), :1))
You could wrap something like DotNetCoords in a SQL CLR function to do this.
See here:- http://www.doogal.co.uk/dotnetcoords.php
I've wrapped it in a CLR function to convert coordinates from Easting/Northing to Lat/Long which I think is what you are asking for. Once the CLR function is implemented it is a pure SQL solution (i.e. you can run it all in a Stored Procedure or View).
EDIT: I will post some sample code up here when I get to work tomorrow, hopefully it will help.
EDIT: You'll need to download the source code from http://www.doogal.co.uk/dotnetcoords.php and you will need Visual Studio to open and modify it. Documentation for the library is here http://www.doogal.co.uk/Help/Index.html
What you can do then is you can add a new class to the source files similar to this:-
using System;
using System.Collections;
using System.Collections.Generic;
using System.Data.SqlTypes;
using DotNetCoords;
using Microsoft.SqlServer.Server;
/// <summary>
/// Sql Server CLR functions for the DotNetCoords library.
/// </summary>
public class CLRFunctions
{
/// <summary>
/// Coordinateses the enumerable.
/// </summary>
/// <param name="Easting">The easting.</param>
/// <param name="Northing">The northing.</param>
/// <returns></returns>
private static IEnumerable<OSRef> CoordinatesEnumerable(double Easting, double Northing)
{
return new List<OSRef> { new OSRef(Easting,Northing) };
}
/// <summary>
/// Toes the lat long.
/// </summary>
/// <param name="Easting">The easting.</param>
/// <param name="Northing">The northing.</param>
/// <returns></returns>
[SqlFunction(FillRowMethodName = "FillRow")]
public static IEnumerable ToLatLong(double Easting, double Northing)
{
return CoordinatesEnumerable(Easting, Northing);
}
/// <summary>
/// Fills the row.
/// </summary>
/// <param name="obj">The obj.</param>
/// <param name="Lat">The lat.</param>
/// <param name="Long">The long.</param>
private static void FillRow(Object obj, out SqlDouble Lat, out SqlDouble Long)
{
OSRef Coordinates = (OSRef)obj;
LatLng latlong = Coordinates.ToLatLng();
latlong.ToWGS84();
Lat = new SqlDouble(latlong.Latitude);
Long = new SqlDouble(latlong.Longitude);
}
}
You will then need to build and import the assembly into SQL Server (replace paths with your own locations) (for some reason I cannot get the assembly to install when PERMISSION_SET is 'SAFE' so I would sort this first before installing in a production environment).
CREATE ASSEMBLY DotNetCoords
FROM N'C:\Projects\DotNetCoords\bin\Debug\DotNetCoords.dll'
WITH PERMISSION_SET = UNSAFE
GO
You'll then need to create a SQL Server function to interface to the CLR function:-
CREATE FUNCTION dbo.ToLatLong(#Easting float, #Northing float)
RETURNS TABLE
(Latitude float null, Longitude float null) with execute as caller
AS
EXTERNAL NAME [DotNetCoords].[CLRFunctions].[ToLatLong]
This is the CLR function installed then.
You should then be able to call the function direct from SQL Server to do your conversion (I have mixed up the numbers in this post too keep anonymity so they might not make sense here but the function does work fine).
/*------------------------
SELECT Latitude, Longitude FROM dbo.ToLatLong(327262, 357394)
------------------------*/
Latitude Longitude
52.13413530182533 -9.34267170569508
(1 row(s) affected)
To use it in a resultset you need to use the CROSS APPLY clause:-
/*------------------------
SELECT TOP 2 a.[Column 0] AS osaddessp,
a.[Column 9] AS east,
a.[Column 10] AS north,
c.[Latitude] AS lat,
c.[Longitude] AS long
FROM MyTable AS a CROSS APPLY ToLatLong (a.[Column 9], a.[Column 10]) AS c;
------------------------*/
osaddessp east north lat long
100134385607 327862 334794 52.3434530182533 -2.19342342569508
100123433149 780268 353406 52.3453417606796 -3.19252323679263
(10 row(s) affected)
Unfortunately, this simply isn't possible. SQL Server Spatial Tools provides a few reprojection functions, but they are only for a very few number of projections (and not the one you require).
There is an example from SQL server tools -- https://bitbucket.org/geographika/sql-server-spatial-tools/src/5ca44b55d3f3/SQL%20Scripts/projection_example.sql -- but it won't help you because they don't support the projection you are talking about.
So, you'll need to adopt a different solution -- either pre-process the data to add a new column with projected values, or reproject in your code.
Related
Per this Using System.Text.Json to Serialize an IConfiguration back to Json it seems that the limited type information that you can put into JSON is discarded.
You seem to be under the impression that IConfiguration objects are storing ints, bools, etc. (for example) corresponding to the JSON Element type. This is incorrect. All data within an IConfiguration is stored in stringified form. The base Configuration Provider classes all expect an IDictionary<string, string> filled with data. Even the JSON Configuration Providers perform an explicit ToString on the values.
I noticed this when I addressed the problem posed in that question with this extension method.
using System.Collections.Generic;
using System.Dynamic;
using Microsoft.Extensions.Configuration;
public static class ExtendConfig
{
public static dynamic AsDynamic(this IConfigurationRoot cr)
{
var result = new ExpandoObject();
var resultAsDict = result as IDictionary<string, object>;
foreach (var item in cr.AsEnumerable())
{
resultAsDict.Add(item.Key, item.Value);
}
return result;
}
}
This method reconstructs the graph but everything is now a string.
I could write my own parser and apply it to the original JSON string but that's a bit dire. Is there any way I can get this metadata so I can improve the fidelity of merged configs? I'm passing it through for consumption by JS which does notice the difference.
Merging is why I'm using the config extensions builder.
As IConfiguration doesn't provide the information about types, but System.Text.Json used by JsonConfigurationProvider does, the working solution (or a workaround) would be using the System.Text.Json deserializer directly to read the configuration file and match the types to the configuration keys.
But we have some minor issues to solve first. Like - where is the configuration file? We don't want to duplicate that information in code, we have to extract it from IConfiguration instance.
Then - match the concrete existing configuration key to the JSON document tree node. That will require either DFS or BFS tree traversal algorithm. I'll go for DFS (Depth First Search). In a nutshell - if you have expandable nodes, you put them in the stack in reverse order. Then you have a while loop that takes a node from the stack, if it has children, you put them on the same stack, if not - you just yield the node. As simple as that, and BFS is pretty similar, but nevermind.
There's one thing more: Newtonsoft.Json - a popular Nuget package that was at a time used even by Microsoft. That JSON serializer is a little slower than System.Text.Json, but it's more advanced, allowing the user to build a JSON document tree node by node.
Having this powerful tool makes creating a writeable JSON IConfiguration relatively easy, especially using some helpers like mine below.
Look at the SaveChanges() method. It walks through the IConfiguration nodes, matches appropriate JObject nodes by their path and copies the changes from IConfiguration instance to the JObject instance. Then you can just write the JSON file.
There's an ugly hack used to get the file. I get the private field containing the IConfigurationRoot instance, but that could be skipped if you already have the configuration root. Having the root you can get JsonConfigurationProvider from it, then it's just Source.Path property.
So here's the code. It's a part of the Woof.Toolkit and Woof.Config Nuget package, that provides writeable JSON configurations, some helper methods to them, and also Azure Key Vault client that uses a JSON configuration, with some helper methods to encrypt and decrypt sensitive data with keys stored on AKV.
This is the first release of ConfigurationExtensions class, so it might be sub-optimal in performance terms, but it works and illustrates how you can match the IConfiguration instance nodes with JObject nodes to get the types of the configuration properties.
using System;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Text.RegularExpressions;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Configuration.Json;
using Newtonsoft.Json.Linq;
namespace Woof.Config;
/// <summary>
/// Extensions for <see cref="IConfiguration"/> making JSON type <see cref="IConfiguration"/> writeable.
/// </summary>
public static class ConfigurationExtensions {
/// <summary>
/// Gets the configuration root element.
/// </summary>
/// <param name="configuration">Any <see cref="IConfiguration"/> part.</param>
/// <returns>Root element.</returns>
public static IConfigurationRoot? GetRoot(this IConfiguration configuration) {
if (configuration is IConfigurationRoot root) return root;
var rootField = configuration.GetType().GetField("_root", BindingFlags.Instance | BindingFlags.NonPublic);
return rootField?.GetValue(configuration) as IConfigurationRoot;
}
/// <summary>
/// Gets the first <see cref="JsonConfigurationProvider"/> if exists, null otherwise.
/// </summary>
/// <param name="root">Configuration root element.</param>
/// <returns><see cref="JsonConfigurationProvider"/> or null.</returns>
public static JsonConfigurationProvider? GetJsonConfigurationProvider(this IConfigurationRoot root)
=> root.Providers.OfType<JsonConfigurationProvider>().FirstOrDefault();
/// <summary>
/// Gets the first <see cref="JsonConfigurationProvider"/> if exists, null otherwise.
/// </summary>
/// <param name="config">Any <see cref="IConfiguration"/> part.</param>
/// <returns><see cref="JsonConfigurationProvider"/> or null.</returns>
public static JsonConfigurationProvider? GetJsonConfigurationProvider(this IConfiguration config)
=> config.GetRoot()?.GetJsonConfigurationProvider();
/// <summary>
/// Saves changes made to <see cref="IConfiguration"/> to the JSON file if exists.
/// </summary>
/// <param name="config">Any <see cref="IConfiguration"/> part.</param>
/// <exception cref="InvalidOperationException">Configuration does not have <see cref="JsonConfigurationProvider"/>.</exception>
public static void SaveChanges(this IConfiguration config) {
var provider = config.GetJsonConfigurationProvider();
if (provider is null) throw new InvalidOperationException("Can't get JsonConfigurationProvider");
var sourceJson = File.ReadAllText(provider.Source.Path);
var target = JObject.Parse(sourceJson);
var stack = new Stack<IConfigurationSection>();
foreach (IConfigurationSection section in config.GetChildren().Reverse()) stack.Push(section);
while (stack.TryPop(out var node)) {
var children = node.GetChildren();
if (children.Any()) foreach (var child in children.Reverse()) stack.Push(child);
else {
var jPath = GetJPath(node.Path);
var element = target.SelectToken(jPath);
var valueString =
element!.Type == JTokenType.Null
? "null" :
element!.Type == JTokenType.String ? $"\"{node.Value}\"" : node.Value;
element!.Replace(JToken.Parse(valueString));
}
}
File.WriteAllText(provider.Source.Path, target.ToString());
}
/// <summary>
/// Sets <paramref name="configuration"/>'s <paramref name="key"/> with specified <paramref name="value"/>.
/// </summary>
/// <param name="configuration">The configuration.</param>
/// <param name="key">The key of the configuration section.</param>
/// <param name="value">Value to set.</param>
/// <exception cref="InvalidOperationException">Not supported type as value.</exception>
public static void SetValue(this IConfiguration configuration, string key, object? value) {
var c = CultureInfo.InvariantCulture;
var valueString = value switch {
null => null,
string v => v,
Uri v => v.ToString(),
byte[] v => Convert.ToBase64String(v),
bool v => v.ToString(c),
int v => v.ToString(c),
decimal v => v.ToString(c),
double v => v.ToString(c),
uint v => v.ToString(c),
long v => v.ToString(c),
ulong v => v.ToString(c),
short v => v.ToString(c),
ushort v => v.ToString(c),
byte v => v.ToString(c),
sbyte v => v.ToString(c),
float v => v.ToString(c),
_ => throw new InvalidOperationException($"Cannot set value of type {value.GetType()}")
};
configuration[key] = valueString;
}
/// <summary>
/// Gets the path for JObject.SelectToken method.
/// </summary>
/// <param name="path"><see cref="IConfiguration"/> path.</param>
/// <returns><see cref="JObject"/> path.</returns>
private static string GetJPath(string path) => RxIConfigurationIndex.Replace(path, "[$1]").Replace(':', '.');
/// <summary>
/// Matches the <see cref="IConfiguration"/> indices.
/// </summary>
private static readonly Regex RxIConfigurationIndex = new(#":(\d+)", RegexOptions.Compiled);
}
Why JObject? Can JSON file represent only an object? No - it can represent any value, including null. But the JSON configuration MUST be an object. That's why I use JObject as my secondary configuration root.
I have a this model:
public partial class UserPosition
{
...
public DbGeometry Position { get; set; }
...
}
I create this object and save it to the MySql DB (db is my DBContext):
var pos = new UserPosition
{
...
Position = GeographyHelper.CreatePoint(request.Latitude, request.Longitude),
...
};
db.UserPositions.Add(pos);
db.SaveChanges();
GeographyHelper.CreatePoint:
/// <summary>
/// Creates a DbGeometry point from latitude and longitude values
/// </summary>
/// <param name="latitude"></param>
/// <param name="longitude"></param>
/// <returns></returns>
public static DbGeometry CreatePoint(double latitude, double longitude)
{
var text = string.Format(CultureInfo.InvariantCulture.NumberFormat, "POINT({0} {1})", longitude, latitude);
// 4326 is most common coordinate system used by GPS/Maps
return DbGeometry.PointFromText(text, 4326);
}
The value is perfectly saved to to DB:
POINT (19.038934 47.531)
But when I read the value from the DB, the Position is null, other fields are OK:
var rec = db.UserPositions.FirstOrDefault()
I get the right record because the ID matches.
Why is the Position null?
Visual Studio 2015, .NET 4.5.2, EntityFramework 6.2.0, MySql.Data.Entity.EF6 6.9.12, MySql database version: 5.6.15.
It was globalization problem: decimal point was saved to the DB but couldn't parse the same geometry data when read because decimal comma was needed (for Double.TryParse()) because of the globalization. The fast solution was to set the globalization to en-US in web.config but I accept if you know better solution :)
<configuration>
<system.web>
<globalization uiCulture="en-US" culture="en-US"/>
...
I was splitting RenameFolder to two pieces and i notice visual studios 2010 supports named parameters! (example below).
I know this has existed for a number of years in other languages. I remember a professor saying why he likes named parameters and that he uses them in all of his code. But i am sure its not in all of his code. I was wondering.
When should i consider to write the function using a named parameter vs normal style (func(a,b,c)). Some cases area (without a when and when not suggestion)
Calling public methods in the same class
Calling private methods in the same class
Calling methods in external 3rd party libraries
Calling methods in another class in the same namespace
Calling methods in another class in a different namespace or module
Calling methods from internal classes not meant to be a library
public bool RenameFolderIn(PK folderId, string newfoldername)
{
return RenameFolder(newfoldername: newfoldername, infolder: true, folderId: folderId);
}
public bool RenameFolderOut(PK folderId, string newfoldername)
{
return RenameFolder(newfoldername: newfoldername, infolder: false, folderId: folderId);
}
public bool RenameFolder(PK folderId, string newfoldername, bool infolder)
{
Typically, I use named parameters when there are a large number of default values and I only need to specify a few non-default, or when the function name doesn't suggest the order of parameters. In the example RenameFolder* functions, I would expect the folder to come before the new name (RenameFolder can be short for the phrase "rename folder folder to name"; phrasing it so the name comes first, if possible, isn't the obvious approach), and so wouldn't bother with named parameters.
Example: suppose Gamma is a constructor for the Gamma distribution, which hase two parameters: shape and scale. There's a statistical convention for passing shape before scale, but the convention isn't obvious from the name, so we use named parameters.
waitTime = Gamma(shape: 2, scale: 2)
I have found an extension method that handles sorting and paging for LINQ. While this works well, I am trying to see if there are some other ways I can use this.
At present, the code for the extension method is as follows:
public static IQueryable<T> Page<T, TResult>(
this IQueryable<T> obj,
int page,
int pageSize,
System.Linq.Expressions.Expression<Func<T, TResult>> keySelector,
bool asc,
out int rowsCount)
{
rowsCount = obj.Count();
int innerRows = (page - 1) * pageSize;
if (asc)
return obj.OrderBy(keySelector).Skip(innerRows).Take(pageSize).AsQueryable();
else
return obj.OrderByDescending(keySelector).Skip(innerRows).Take(pageSize).AsQueryable();
}
The method takes in an expression, which is based off the type.
In my Dealer class, I have a method GetDealers, which essentially calls this,
i.e.
db.User.Page(1, 2, p => p.User.UserProperty.Name, true, out rowCount)
From the presentation side of things though, I do not know or can access the expression as above, e.g.
ListView1.DataSource = users.GetDealers("SortColumn", pageNo, pageSize, out rowCount, bool asc);
ListView1.DataBind();
The only way is to have a switch statement in my GetDealers method that would then convert to the expression. Is there a way to bypass this, or is this method OK?
I'm not exactly sure what you're asking, but I believe it's something that I have looked into myself. If you would like to know how to dynamically sort your results based on a string, rather than a proper LINQ expression, then you're in luck.
Scott Guthrie published a great article on that very topic. It references a Microsoft file which extends any IQueryable object to support dynamic sorting.
C# Dynamic Query Library (included in the \LinqSamples\DynamicQuery directory). Just add the page to your App_Code folder and include "Using System.Linq.Dynamic" in your project and you will be able to use the following syntax:
myUsers = myUsers.OrderBy("LastName");
I hope this helps!
If you are looking for extension method to work on all types
public static class SortingAndPagingHelper
{
/// <summary>
/// Returns the list of items of type on which method called
/// </summary>
/// <typeparam name="TSource">This helper can be invoked on IEnumerable type.</typeparam>
/// <param name="source">instance on which this helper is invoked.</param>
/// <param name="sortingModal">Page no</param>
/// <returns>List of items after query being executed on</returns>
public static IEnumerable<TSource> SortingAndPaging<TSource>(this IEnumerable<TSource> source, SortingAndPagingInfo sortingModal)
{
// Gets the coloumn name that sorting to be done o`enter code here`n
PropertyInfo propertyInfo = source.GetType().GetGenericArguments()[0].GetProperty(sortingModal.SortColumnName);
// sorts by ascending if sort criteria is Ascending otherwise sorts descending
return sortingModal.SortOrder == "Ascending" ? source.OrderByDescending(x => propertyInfo.GetValue(x, null)).Skip(sortingModal.PageSelected * sortingModal.PageSize).Take(sortingModal.PageSize)
: source.OrderBy(x => propertyInfo.GetValue(x, null)).Skip(sortingModal.PageSelected * sortingModal.PageSize).Take(sortingModal.PageSize);
}
}
DbContext dbContext = new DbContext();
dbContext.rainingSessions.Where(x => x.RegistrationDeadline > DateTime.Now) .SortingAndPaging(sortAndPagingInfo).ToList()
Sql Server 2008 supports spatial data with new geometry and geography UDT's. They both support AsGml() method to serialize data in gml format. However they serialize data into GML3 format. Is there any way to tell it to serialize data into GML2 format?
AFAIK, there is no built-in feature to serialize geospatial data to GML 2.x. You need to use some third-party tools, implement a writer youserlf or, this suggestion may sound a bit strange, use PostGIS for this transition.
PostGIS is alternative geospatial database, similar solution to SQL Server, but implementing de-/serialization for both formats: GML 2 and GML 3.
What I'd suggest is to use PostGIS as an intermediate and translating storage.
Store data to GML 3 using SQL Server functions
Load data serialized to GML 3 using PostGIS function ST_GeomFromGML
Store data from PostGIS to GML 2 format using ST_AsGML which allows you to specify target version of GML: text ST_AsGML(integer version, geometry g1);
It may sound strange to propose another geospatial database, but I'm sure it would work fairly smoothly and well.
There is no support for GML2, but there is extensibility API that can be used to implement custom serialization.
Here is an example of custom serialization using SqlGeometry.Populate(IGeometrySink) method (C# code):
CustomWriter w = new CustomWriter();
SqlGeometry.Parse("POLYGON ((0 0, 10 0, 10 10, 0 10, 0 0))").Populate(w);
System.Console.WriteLine(w);
public class CustomWriter : IGeometrySink {
private StringBuilder _builder = new StringBuilder();
public string ToString() {
return _builder.ToString();
}
public void SetSrid(int srid) {
_builder.Append('#');
_builder.Append(srid);
}
public void BeginGeometry(OpenGisGeometryType type) {
_builder.Append(" (");
_builder.Append(type);
}
public void BeginFigure(double x, double y, double? z, double? m) {
_builder.Append(" [");
_builder.Append(x);
_builder.Append(' ');
_builder.Append(y);
}
public void AddLine(double x, double y, double? z, double? m) {
_builder.Append(',');
_builder.Append(x);
_builder.Append(' ');
_builder.Append(y);
}
public void EndFigure() {
_builder.Append(']');
}
public void EndGeometry() {
_builder.Append(')');
}
}
To do deserialization use SqlGeometryBuilder class:
// Create "POLYGON ((0 0, 10 0, 10 10, 0 10, 0 0))" using Builder API
SqlGeometryBuilder b = new SqlGeometryBuilder();
b.SetSrid(0);
b.BeginGeometry(OpenGisGeometryType.Polygon);
b.BeginFigure(0, 0);
b.AddLine(10, 0);
b.AddLine(10, 10);
b.AddLine(0, 10);
b.AddLine(0, 0);
b.EndFigure();
b.EndGeometry();
SqlGeometry g = b.ConstructedGeometry;
As Marko said, there is no support for gml2 in Sql Server 2008, so I just ended up writing a function for transforming gml3 returned by the server to gml2 that I needed.
Well since you finished there is not much point but I would recommend putting geoserver in front of SQL Server. Geoserver has all the serialization code built in for almost any format you want, is easy cheezy to install, and works as advertised.
http://docs.geoserver.org/2.0.x/en/user/services/wfs/outputformats.html