'__post_init__' or dynamically create new fields of SQLModel after initiation - sqlalchemy

My objective is to create a new field based on another field after a Request is posted to FastAPI. My attempt was:
import functools
from datetime import date, datetime
from slugify import slugify
from sqlmodel import Field, SQLModel
class ProjectBase(SQLModel):
is_active: bool = Field(default=True, nullable=False)
name: str = Field(..., nullable=False)
publish_date: date = Field(default_factory=datetime.now().date, nullable=False)
# # post_init
repr_name: str = Field(
default_factory=functools.partial(slugify, name, separator='_'),
description="The project represented name, it must contain no whitespace, each word is separated by an underscore and it is slugified using the python-slugify library.",
nullable=False)
I have also tried __post_init__ but I think SQLModel does not have such a mechanism, it belongs to dataclasses within pydantic.
My desired output would be something like if a Request like the below was POST-ed:
request = {
'is_active': true,
'name': 'hello world bye',
'publish_date': '2023-01-01'
}
Then, the following Response is gotten and inserted into the database:
response = {
'is_active': true,
'name': 'hello world bye',
'repr_name': 'hello_world_bye', # output of slugify(`name`, separator='_')
'publish_date': '2023-01-01'
}

Thankfully sqlmodel.SQLModel inherits from pydantic.BaseModel. (And their metaclasses are also related.)
This is yet another job for custom validators. We just need a default sentinel value to check against, if no explicit value is provided by the user. I would just make the field type a union of str and None and make None the default, but then always ensure a str ends up as the value via the validator.
Here is an example:
from datetime import date, datetime
from typing import Any, Optional
from pydantic import validator
from slugify import slugify
from sqlmodel import Field, SQLModel
class ProjectBase(SQLModel):
name: str = Field(..., nullable=False)
publish_date: date = Field(default_factory=datetime.now().date, nullable=False)
repr_name: Optional[str] = Field(default=None, nullable=False)
#validator("repr_name", always=True)
def slugify_name(cls, v: Optional[str], values: dict[str, Any]) -> str:
if v is None:
return slugify(values["name"], separator="_")
return slugify(v, separator="_")
print(ProjectBase(name="foo bar baz").json(indent=2))
print(ProjectBase(name="spam", repr_name="Eggs and Bacon").json(indent=2))
Output:
{
"name": "foo bar baz",
"publish_date": "2023-02-11",
"repr_name": "foo_bar_baz"
}
{
"name": "spam",
"publish_date": "2023-02-11",
"repr_name": "eggs_and_bacon"
}
The always=True is important to trigger validation, when no value was explicitly provided.
The order of the fields is also important because the values dictionary in the validator method will only contain previously validated fields and field validation occurs in the order the fields were defined (see docs link above), so since name comes before repr_name, it will be validated and its value therefore present in the dictionary, when repr_name validation is triggered.
Important: Validation will not work, if the validators belong to a table=True model. To get around this, define a base model with the fields you need validated and inherit from that in your table=True model; then parse data through the parent model before instantiating the table model with it.
This bit seems pretty limiting, so I would not rule out the possibility that this will change in the future, at least to a certain extent. But in the current state of SQLModel that is what we have.

Related

It's a bad design to try to print classes' variable name and not value (eg. x.name print "name" instead of content of name)

The long title contain also a mini-exaple because I couldn't explain well what I'm trying to do. Nonethless, the similar questions windows led me to various implementation. But since I read multiple times that it's a bad design, I would like to ask if what I'm trying to do is a bad design rather asking how to do it. For this reason I will try to explain my use case with a minial functional code.
Suppose I have a two classes, each of them with their own parameters:
class MyClass1:
def __init__(self,param1=1,param2=2):
self.param1=param1
self.param2=param2
class MyClass2:
def __init__(self,param3=3,param4=4):
self.param3=param3
self.param4=param4
I want to print param1...param4 as a string (i.e. "param1"..."param4") and not its value (i.e.=1...4).
Why? Two reasons in my case:
I have a GUI where the user is asked to select one of of the class
type (Myclass1, Myclass2) and then it's asked to insert the values
for the parameters of that class. The GUI then must show the
parameter names ("param1", "param2" if MyClass1 was chosen) as a
label with the Entry Widget to get the value. Now, suppose the
number of MyClass and parameter is very high, like 10 classes and 20
parameters per class. In order to minimize the written code and to
make it flexible (add or remove parameters from classes without
modifying the GUI code) I would like to cycle all the parameter of
Myclass and for each of them create the relative widget, thus I need
the paramx names under the form od string. The real application I'm
working on is even more complex, like parameter are inside other
objects of classes, but I used the simpliest example. One solution
would be to define every parameter as an object where
param1.name="param1" and param1.value=1. Thus in the GUI I would
print param1.name. But this lead to a specifi problem of my
implementation, that's reason 2:
MyClass1..MyClassN will be at some point printed in a JSON. The JSON
will be a huge file, and also since it's a complex tree (the example
is simple) I want to make it as simple as possibile. To explain why
I don't like to solution above, suppose this situation:
class MyClass1:
def init(self,param1,param2,combinations=[]):
self.param1=param1
self.param2=param2
self.combinations=combinations
Supposse param1 and param2 are now list of variable size, and
combination is a list where each element is composed by all the
combination of param1 and param2, and generate an output from some
sort of calculation. Each element of the list combinations is an
object SingleCombination,for example (metacode):
param1=[1,2] param2=[5,6] SingleCombination.param1=1
SingleCombination.param2=5 SingleCombination.output=1*5
MyInst1.combinations.append(SingleCombination).
In my case I will further incapsulated param1,param2 in a object
called parameters, so every condition will hace a nice tree with
only two object, parameters and output, and expanding parameters
node will show all the parameters with their value.
If I use JSON pickle to generate a JSON from the situation above, it
is nicely displayed since the name of the node will be the name of
the varaible ("param1", "param2" as strings in the JSON). But if I
do the trick at the end of situation (1), creating an object of
paramN as paramN.name and paramN.value, the JSON tree will become
ugly but especially huge, because if I have a big number of
condition, every paramN contains 2 sub-element. I wrote the
situation and displayed with a JSON Viewer, see the attached immage
I could pre processing the data structure before creating the JSON,
the problem is that I use the JSON to recreate the data structure in
another session of the program, so I need all the pieces of the data
structure to be in the JSON.
So, from my requirements, it seems that the workround to avoid print the variable names creates some side effect on the JSON visualization that I don't know how to solve without changing the logic of my program...
If you use dataclasses, getting the field names is pretty straightforward:
from dataclasses import dataclass, fields
#dataclass
class MyClass1:
first:int = 4
>>> fields(MyClass1)
(Field(name='first',type=<class 'int'>,default=4,...),)
This way, you can iterate over the class fields and ask your user to fill them. Note the field has a type, which you could use to eg ask the user for several values, as in your example.
You could add functions to extract programatically the param names (_show_inputs below ) from the class and values from instances (_json below ):
def blossom(cls):
"""decorate a class with `_json` (classmethod) and `_show_inputs` (bound)"""
def _json(self):
return json.dumps(self, cls=DataClassEncoder)
def _show_inputs(cls):
return {
field.name: field.type.__name__
for field in fields(cls)
}
cls._json = _json
cls._show_inputs = classmethod(_show_inputs)
return cls
NOTE 1: There's actually no need to decorate the classes with blossom. You could just use its internal functions programatically.
Using a custom json encoder to dump the dataclass objects, including properties:
import json
class DataClassPropEncoder(json.JSONEncoder): # https://stackoverflow.com/a/51286749/7814595
def default(self, o):
if is_dataclass(o):
cls = type(o)
# inject instance properties
props = {
name: getattr(o, name)
for name, value in cls.__dict__.items() if isinstance(value, property)
}
return {
**props,
**asdict(o)
}
return super().default(o)
Finally, wrap the computations inside properties so they are
serialized as well when using the decorated class. Full code example:
from dataclasses import asdict
from dataclasses import dataclass
from dataclasses import fields
from dataclasses import is_dataclass
import json
from itertools import product
from typing import List
class DataClassPropEncoder(json.JSONEncoder): # https://stackoverflow.com/a/51286749/7814595
def default(self, o):
if is_dataclass(o):
cls = type(o)
props = {
name: getattr(o, name)
for name, value in cls.__dict__.items() if isinstance(value, property)
}
return {
**props,
**asdict(o)
}
return super().default(o)
def blossom(cls):
def _json(self):
return json.dumps(self, cls=DataClassEncoder)
def _show_inputs(cls):
return {
field.name: field.type.__name__
for field in fields(cls)
}
cls._json = _json
cls._show_inputs = classmethod(_show_inputs)
return cls
#blossom
#dataclass
class MyClass1:
param1:int
param2:int
#blossom
#dataclass
class MyClass2:
param3: List[str]
param4: List[int]
def _compute_single(self, values): # TODO: implmement this
return values[0]*values[1]
#property
def combinations(self):
# TODO: cache if used more than once
# TODO: combinations might explode
field_names = []
field_values = []
cls = type(self)
for field in fields(cls):
field_names.append(field.name)
field_values.append(getattr(self, field.name))
results = []
for values in product(*field_values):
result = {
**{
field_names[idx]: value
for idx, value in enumerate(values)
},
"output": self._compute_single(values)
}
results.append(result)
return results
>>> print(f"MyClass1:\n{MyClass1._show_inputs()}")
MyClass1:
{'param1': 'int', 'param2': 'int'}
>>> print(f"MyClass2:\n{MyClass2._show_inputs()}")
MyClass2:
{'param3': 'List', 'param4': 'List'}
>>> obj_1 = MyClass1(3,4)
>>> print(f"obj_1:\n{obj_1._json()}")
obj_1:
{"param1": 3, "param2": 4}
>>> obj_2 = MyClass2(["first", "second"],[4,2])._json()
>>> print(f"obj_2:\n{obj_2._json()}")
obj_2:
{"combinations": [{"param3": "first", "param4": 4, "output": "firstfirstfirstfirst"}, {"param3": "first", "param4": 2, "output": "firstfirst"}, {"param3": "second", "param4": 4, "output": "secondsecondsecondsecond"}, {"param3": "second", "param4": 2, "output": "secondsecond"}], "param3": ["first", "second"], "param4": [4, 2]}
NOTE 2: If you need to perform several computations per class, it might be a good idea to abstract away the pattern in the combinations property to avoid repeating code.
NOTE 3: If you need access to the properties several times and not ust once, you might want to consider caching their values to avoid re-computation.
Once you have an instance of MyClass / MyClass2, you can call vars() or vars().keys() and it will give you the attributes as a str. Unlike dir, it will not show all the builtin attributes/methods starting with __.
class MyClass2:
def __init__(self,param3=3,param4=4):
self.param3=param3
self.param4=param4
instance_of_myclass2 = MyClass2(param3="what", param4="ever")
print(vars(instance_of_myclass2))
{'param3': 'what', 'param4': 'ever'}
print(vars(instance_of_myclass2).keys())
dict_keys(['param3', 'param4'])
dir(instance_of_myclass2)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'param3', 'param4']

marshmallow schema validation dash problem

I'm using marshmallow for validate some fields of a schema.
class PlantDetailsSchema(Schema):
name: fields.Str(required=True, validate=validate.Length(min=3)),
sprout-time: fields.Str(required=True, validate=validate.Length(min=3)),
full-growth: fields.Str(required=True, validate=validate.Length(min=5)),
edible: fields.Bool(required=True)
class PlantInfoSchema(Schema):
plant = fields.Nested(PlantDetailsSchema)
num = fields.Int(required=True, validate=validate.Range(min=1))
def validate_json(json):
try:
schema = PlantInfoSchema().load(json)
ret = schema.dump()
return ret
except ValidationError:
return None
The json element to validate is:
json = {'plant': {'name': 'Bonsai', 'sprout-time': '3 months', 'full-growth': '2 years', 'edible': False}, 'num': 3}
My issue is how to validate fields that contains a dash in the middle of the name (like 'sprout-time' and 'full-growth'. Do you have some ideas to solve this issue?
You can use the 'data_key' attribute to specify the key name with dash. Define the field name something which is valid like 'sprout_time' (could be anything) and data_key as an attribute to the field with the actual name 'sprout-time'.
class PlantDetailsSchema(Schema):
name = fields.Str(required=True, validate=validate.Length(min=3))
sprout_time = fields.Str(
data_key='sprout-time',
required=True, validate=validate.Length(min=3)
)
full_growth = fields.Str(
data_key='full-growth',
required=True,
validate=validate.Length(min=5)
)
edible = fields.Bool(required=True)

How to use keyTextTransform() for nested json?

My model has a json field. I can access jsonfield['key1'] with the following query
from django.contrib.postgres.fields.jsonb import KeyTextTransform
MyModel.objects.annotate(val=KeyTextTransform('key1', 'jsonfield')).order_by('val')
But how can I access a key like jsonfield['key1']['key2'] or even more nested ones?
This can't be the only solution, right?
MyModel.objects.annotate(val=KeyTextTransform('key2', (KeyTextTransform('key1', 'jsonfield'))).order_by('val')
The hard part is already done, thankfully. KeyTextTransform is composable. All we have to do is compose it.
class NestableKeyTextTransform:
def __new__(cls, field, *path):
if not path:
raise ValueError("Path must contain at least one key.")
head, *tail = path
field = KeyTextTransform(head, field)
for head in tail:
field = KeyTextTransform(head, field)
return field
MyModel.objects.annotate(
single_nested_value=NestableKeyTextTransform(
"json_field", "query", "name"
),
array_access=NestableKeyTextTransform(
"json_field", "query", "address_line", 1
),
)
Though I would like to point out that this may be a better way to do it:
from django.db.models import F
MyModel.objects.annotate(
single_nested_value=F("json_field__query__name"),
array_access=F("json_field__query__address_line__1"),
)
NB as of the start of 2023 (in django's development version) you can now also do:
from django.db.models.fields.json import KT
Dogs.objects.annotate(
first_breed=KT("data__breed__1"),
owner_name=KT("data__owner__name")
)
See docs and feature request

filter particular field name and value from field_dict of package django-reversion

I have a function which returns json data as history from Version of reversion.models.
from django.http import HttpResponse
from reversion.models import Version
from django.contrib.admin.models import LogEntry
import json
def history_list(request):
history_list = Version.objects.all().order_by('-revision__date_created')
data = []
for i in history_list:
data.append({
'date_time': str(i.revision.date_created),
'user': str(i.revision.user),
'object': i.object_repr,
'field': i.revision.comment.split(' ')[-1],
'new_value_field': str(i.field_dict),
'type': i.content_type.name,
'comment': i.revision.comment
})
data_ser = json.dumps(data)
return HttpResponse(data_ser, content_type="application/json")
When I run the above snippet I get the output json as
[{"type": "fruits", "field": "colour", "object": "anyobject", "user": "anyuser", "new_value_field": "{'price': $23, 'weight': 2kgs, 'colour': 'red'}", "comment": "Changed colour."}]
From the function above,
'comment': i.revision.comment
returns json as "comment": "changed colour" and colour is the field which I have written in the function to retrieve it from comment as
'field': i.revision.comment.split(' ')[-1]
But i assume getting fieldname and value from field_dict is a better approach
Problem: from the above json list I would like to filter new_field_value and old_value. In the new_filed_value only value of colour.
Getting the changed fields isn't as easy as checking the comment, as this can be overridden.
Django-reversion just takes care of storing each version, not comparing.
Your best option is to look at the django-reversion-compare module and its admin.py code.
The majority of the code in there is designed to produce a neat side-by-side HTML diff page, but the code should be able to be re-purposed to generate a list of changed fields per object (as there can be more than one changed field per version).
The code should* include a view independent way to get the changed fields at some point, but this should get you started:
from reversion_compare.admin import CompareObjects
from reversion.revisions import default_revision_manager
def changed_fields(obj, version1, version2):
"""
Create a generic html diff from the obj between version1 and version2:
A diff of every changes field values.
This method should be overwritten, to create a nice diff view
coordinated with the model.
"""
diff = []
# Create a list of all normal fields and append many-to-many fields
fields = [field for field in obj._meta.fields]
concrete_model = obj._meta.concrete_model
fields += concrete_model._meta.many_to_many
# This gathers the related reverse ForeignKey fields, so we can do ManyToOne compares
reverse_fields = []
# From: http://stackoverflow.com/questions/19512187/django-list-all-reverse-relations-of-a-model
changed_fields = []
for field_name in obj._meta.get_all_field_names():
f = getattr(
obj._meta.get_field_by_name(field_name)[0],
'field',
None
)
if isinstance(f, models.ForeignKey) and f not in fields:
reverse_fields.append(f.rel)
fields += reverse_fields
for field in fields:
try:
field_name = field.name
except:
# is a reverse FK field
field_name = field.field_name
is_reversed = field in reverse_fields
obj_compare = CompareObjects(field, field_name, obj, version1, version2, default_revision_manager, is_reversed)
if obj_compare.changed():
changed_fields.append(field)
return changed_fields
This can then be called like so:
changed_fields(MyModel,history_list_item1, history_list_item2)
Where history_list_item1 and history_list_item2 correspond to various actual Version items.
*: Said as a contributor, I'll get right on it.

Dynamically define name of class in peewee model

I'm trying to assign a class name dynamically using a string.
Much like this...
classname='cats'
class classname(peewee.Model):
Peewee doesn't seem to think I should be able to do this and I'm having a lot of trouble finding a way to define the class name dynamically.
Help!
If you need to control the table name, you can do:
class MyModel(Model):
whatever = CharField()
class Meta:
db_table = 'my_table_name'
I have a same problem with you,and find a solution at last.
Maybe it's not a good idea to Dynamically define name of class in peewee model.
class Person(Model):
name = CharField()
birthday = DateField()
is_relative = BooleanField()
class Meta:
database = db
db_table = "Hello"
>>> Person.__dict__
mappingproxy({'DoesNotExist': <class 'peewee.PersonDoesNotExist'>, 'is_relative': <peewee.FieldDescriptor object at 0x000000000470BEF0>, 'name': <peewee.FieldDescriptor object at 0x000000000470BF60>, '_meta': <peewee.ModelOptions object at 0x000000000470BE48>, 'birthday': <peewee.FieldDescriptor object at 0x000000000470BF98>, 'id': <peewee.FieldDescriptor object at 0x000000000470BEB8>, '__module__': 'data', '_data': None, '__doc__': None})
You could see clearly what meta class means in the __dict__,and you can change it in this way:
setattr(Person._meta, "db_table", "another")
db.connect()
db.create_table(Person)
I'm not a native English speaker and a newbie in programming,but I hope the solution is helpful for you.
Using the "exec" function would be easiest way of doing this:
CLASS_TEMPLATE = """\
class %s(peeww.Model):
pass
"""
classname = "cats"
exec(CLASS_TEMPLATE % classname)
c = cats() # works !
Explanation:
I've created a string named CLASS_TEMPLATE which contains Python code creating a class. I use a format specifier %s as a class name, so it can be replaced.
Doing this:
CLASS_TEMPLATE % "cats"
replaces the format specifier %s and creates a string that contains the code to create a class named "cats".
If you ask Python to execute this code, your class gets defined:
exec(CLASS_TEMPLATE % "cats")
You can see that it exists in your namespace:
>>> globals()
{..., 'cats': <class __main__.cats at 0x1007b9b48>, ...}
... and that it is a class:
>>> type(cats)
<type 'classobj'>
And you can start using this newly defined class to instantiate objects:
c = cats()