Python lightweight serialization and deserialization package marshmallow detailed usage guide 3 Marshmallow Official documentation: https://marshmallow.readthedocs.io/en/latest/ Marshmallow, Chinese translation: cotton candy. It is a lightweight data format conversion module, also called serialization and deserialization module, which is often used to convert between complex orm model objects and python native data types. Marshmallow provides rich API functions. as follows: Serializing Serialization [can convert data objects into storable or transferable data types, for example: objects/object->list/dict, dict/list->string] Deserializing Deserializer [converts storable or transferable data types into data objects, for example: list/dict->objects/object, string->dict/list] Validation Data verification, you can perform type verification or custom verification on the content of the data to be converted during the deserialization stage. Marshmallow itself is a separate library, based on the framework of our current project is flask and the database ORM framework uses SQLAlchemy, so we can integrate it into the project by installing flask-sqlalchemy and marshmallow-sqlalchemy. Basic installation and configuration Module installation: pip install -U marshmallow-sqlalchemy pip install -U flask-sqlalchemy pip install -U flask-marshmallow Module initialization: import os from flask import Flask from flask_script import Manager from flask_sqlalchemy import SQLAlchemy from flask_redis import FlaskRedis from flask_session import Session from flask_migrate import Migrate,MigrateCommand from flask_jsonrpc import JSONRPC from flask_marshmallow import Marshmallow from application.utils import init_blueprint from application.utils.config import load_config from application.utils.session import init_session from application.utils.logger import Log from application.utils.commands import load_command # Create terminal script management object manager = Manager() # Create database link object db = SQLAlchemy() # redis link object redis = FlaskRedis() # Session storage object session_store = Session() # data migration instance object migrate = Migrate() # log object log = Log() # jsonrpc module instance object jsonrpc = JSONRPC() # Object creation for data converters ma = Marshmallow() def init_app(config_path): """Global initialization""" # create app application object app = Flask(__name__) # Project root directory app.BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) # load configuration Config = load_config(config_path) app.config.from_object(Config) # Database initialization db.init_app(app) redis.init_app(app) # Initialization of the data converter ma.init_app(app) # session storage initialization init_session(app) session_store.init_app(app) # Data migration initialization migrate.init_app(app, db) # Add data migration commands to the terminal script tool manager.add_command('db', MigrateCommand) # log initialization app.log = log.init_app(app) # Blueprint Registration init_blueprint(app) # jsonrpc initialization jsonrpc.service_url = "/api" # URL address prefix of api interface jsonrpc.init_app(app) # Initialize terminal script tool manager.app = app # Register custom commands load_command(manager) return manager Basic constructor (Schema) The marshmallow conversion data format is mainly done through the constructor class, while the Schema class provides the basic functions of data conversion: serialization, verification and deserialization. So in the process of using marshmallow, all constructor classes must directly or indirectly inherit from the Schema base class Complete data serialization conversion based on Schema application.apps.marsh.viewsCode: from marshmallow import Schema,fields from application.apps.users.models import User,UserProfile class UserSchema(Schema): # Basic constructor (Schema) '''Conversion of user information has nothing to do with the model fields set field type ''' name = fields. String() age = fields. Integer() email = fields. Email() money = fields. Number() classMeta: fields = ["name","age","money","email","info"] # ('Fields that need to be serialized') ordered = True # Convert to an ordered dictionary def index(): """Serialization""" """Serialization processing of single model data""" user1 = User( name="xiaoming", password="123456", age=16, email="[email protected]", money=31.50 ) # print(user1) # Convert the model object to dictionary format data1 = UserSchema(). dump(user1) print(type(data1),data1) # Convert the model object to json string format data2 = UserSchema().dumps(user1) # dumps serializes any object into a str print(type(data2), data2) return "ok" Schema common attribute data type Type Description fields.Dict(keys, type]] = None, values, …) dictionary type, often used to receive json type data fields. List(cls_or_instance, type], **kwargs) List type, often used to receive array data fields.Tuple (tuple_fields, *args, **kwargs) tuple type fields.String(* , default, missing, data_key, …) String type fields.UUID(*, default, missing, data_key, …) UUID format type string fields.Number(*, as_string, **kwargs) Numeric basic types fields.Integer(*, strict, **kwargs) Integer fields.Decimal(places, rounding, *, allow_nan, …) numeric fields.Boolean(*, truthy, falsy, **kwargs) Boolean fields.Float(*, allow_nan, as_string, **kwargs) floating point type fields .DateTime(format, **kwargs) DateTimeType fields.Time (format, **kwargs) Time type fields.Date(format, **kwargs) DateType fields.Url(*, relative, schemes, Set[str]]] = None, …) url URL string type fields.Email(*args, **kwargs) Email String type fields.IP(*args[, exploded]) IP address string type fields.IPv4(*args[, exploded]) IPv4 address string type fields.IPv6(*args[, exploded]) IPv6 address string type fields .Method(serialize, deserialize, **kwargs) Fields based on Schema class method return values fields.Function(serialize, Any], Callable[[Any, …) Returns value fields based on function fields.Nested(nested, type, str, Callable[[], …) Foreign key type Common common properties of Schema data types Attribute name Description default Set the default value of the field in the serialization phase missing Set the default value of the field in the deserialization phase validate The built-in data validator or built-in validation set called by the deserialization phase required Set the required fields of the current field allow_none Whether it is allowed to be empty load_only Whether the current field is used in the deserialization phase dump_omly Is the current field used in the serialization phase dump_omly Whether the current field is used in the serialization phase error_messages dictionary type, which can be used to replace the default field exception prompt, format:error_messages={“required”: “The username is required.”} The serialization operation performed above belongs to the serialization of a single data object, and the serialization of multiple data objects can also be performed in MarshMallow. from marshmallow import Schema,fields from application.apps.users.models import User,UserProfile class UserSchema(Schema): name = fields. String() age = fields. Integer() email = fields. Email() money = fields. Number() classMeta: fields = ["name","age","money","email","info"] ordered = True # Convert to an ordered dictionary def index(): """Serialization""" """Serialization of multiple model data""" user1 = User(name="xiaoming", password="123456", age=15, email="[email protected]", money=31.50) user2 = User(name="xiaohong", password="123456", age=16, email="[email protected]", money=31.50) user3 = User(name="xiaopang", password="123456", age=17, email="[email protected]", money=31.50) data_list = [user1, user2, user3] data1 = UserSchema(many=True).dumps(data_list) # dumps serializes any object into a str many=True allows multiple models to be serialized print(type(data1),data1) # <class 'str'> [{"name": "xiaoming", "age": 4, "money": 31.5, "email": "33@qq .com", "info": null}, {"name": "xiaohui", "age": 5, "money": 66.5, "email": " "[email protected]", "info": null}, {"name": "xiaobai", "age": 6, "money": 88.5, "email ": "[email protected]", "info": null}] return "ok" Constructor nested use from marshmallow import Schema,fields from application.apps.users.models import User,UserProfile class UserProfileSchema(Schema): education = fields. Integer() middle_school = fields. String() class UserSchema(Schema): name = fields. String() age = fields. Integer() email = fields. Email() money = fields. Number() info = fields.Nested(UserProfileSchema,only=["middle_school"]) # nested foreign key type only only displays a certain field classMeta: fields = ["name","age","money","email","info"] ordered = True # Convert to an ordered dictionary def index(): """Serialization""" """Serialization nested using """ user1 = User(name="xiaoming", password="123456", age=15, email="[email protected]", money=31.50) user1.info = UserProfile( education=3, middle_school="qwq school" ) data = UserSchema().dump(user1) #order sorted dictionary print(data) # OrderedDict([('name', 'xiaoming'), ('age', 15), ('money', 31.5), ('email', \ '[email protected]'), ('info', {'middle_school': 'qwqschool'})]) data1 = UserSchema().dumps(user1) # object to string print(data1) # {"name": "xiaoming", "age": 15, "money": 31.5, "email": "[email protected]", "info": {"middle_school": "qwq\\学\\校"}} return "ok" Complete data deserialization conversion based on Schema code: class UserSchema2(Schema): name = fields.String(required=True,error_messages={<!-- -->"required":"Must be filled in"}) # required=True must be passed parameters, if there is no error sex = fields. String() age = fields.Integer(missing=18) # set the default value when deserializing email = fields.Email(error_messages={<!-- -->'invalid':"Sorry, the email must be formatted correctly!"}) mobile = fields. String() @post_load # Deserialize into an object, add post_load decorator to a method of schema def post_load(self, data, **kwargs): return User(**data) def index(): user_data = {<!-- -->"mobile":"1331345635", "email": "[email protected]","sex":"abc"} us2 = UserSchema2() result = us2.load(user_data,partial=True) # The default serializer must pass all required fields, otherwise a validation exception will be thrown. But we can use partial=True to avoid error print(result) # ==> <User xiaoming> return "ok" Convert/ignore some data when deserializing from marshmallow import Schema, fields, validate, ValidationError, post_load class UserSchema2(Schema): name = fields. String() sex = fields. String() age = fields. Integer(missing=18) email = fields. Email() mobile = fields. String(required=True) @post_load def post_load(self, data, **kwargs): return User(**data) def index(): user_data = {<!-- -->"name": "xiaoming","sex":"abc"} us2 = UserSchema2() result = us2.load(user_data,partial=True) # The default serializer must pass all required fields, otherwise a validation exception will be thrown. But we can use the partial parameter to allow partial field updates print(result) # ==> <User xiaoming> return "ok" The setting field is only enabled during the serialization or deserialization phase class UserSchema2(Schema): name = fields. String() sex = fields.Integer(validate=validate.OneOf([0,1,2])) # Deserialization stage calls selection validation age = fields. Integer(missing=18) email = fields. Email() mobile = fields. String() password = fields.String(load_only=True) # Set the current field as a write-only field, "write-only" will only be enabled in the deserialization phase # password = fields.String(dump_only=True) # Set the current field as a write-only field, "read-only" will only be enabled in the serialization phase @post_load def post_load(self, data, **kwargs): return User(**data) def index(): user_data = {<!-- -->"name": "xiaoming","password":"123456","sex":1} us2 = UserSchema2() # deserialize result = us2. load(user_data) print(result) # ==> <User xiaoming> # Serialization us3 = UserSchema2(only=["sex","name","age","password"]) # Limit processing fields result2 = us3. dump(result) print(result2) return "ok" Hook method in the deserialization phase post_dump([fn, pass_many, pass_original]) Registers a method to be called after serializing the object, it will be called after the object is serialized. post_load([fn, pass_many, pass_original]) Registers the method to be called after deserializing the object, which will be called after validating the data. pre_dump([fn, pass_many]) Registers a method to be called before serializing the object, it will be called before serializing the object. pre_load([fn, pass_many]) Before deserializing the object, register the method to be called, it will be called before validating the data. from marshmallow import Schema, fields, validate, ValidationError, post_load, post_dump class UserSchema2(Schema): name = fields. String() sex = fields.Integer(validate=validate.OneOf([0,1,2])) age = fields. Integer(missing=18) email = fields. Email() mobile = fields. String() password = fields.String(load_only=True) # Set the current field as a write-only field, which will only be enabled in the deserialization phase @post_load def post_load(self, data, **kwargs): return User(**data) @post_dump def post_dump(self, data, **kwargs): data["mobile"] = data["mobile"][:3] + "*****" + data["mobile"][-3:] return data def index(): user_data = {<!-- -->"name": "xiaoming","password":"123456","sex":1,"mobile":" 133123454656"} us2 = UserSchema2() # deserialize result = us2. load(user_data) print(result) # ==> <User xiaoming> # Serialization us3 = UserSchema2(only=["sex","name","age","mobile"]) # Limit the fields processed result2 = us3. dump(result) print(result2) return "ok" Validation of data during deserialization Data validation based on built-in validators Built-in Validator Description validate.Email (*, error) Mailbox validation validate.Equal(comparable, *, error) Judge whether the values are equal validate.Length(min, max, *, equal, error) value length/size validation validate.OneOf(choices, labels, *, error) option validation validate.Range([min, max]) Range validation validate.Regexp(regex, bytes, Pattern][, flags]) regular validation validate. URL(*, relative, schemes, Set[str]]] = None, …) Verify whether it is a URL < /table> code: from marshmallow import Schema, fields, validate, ValidationError, post_load class UserSchema3(Schema): name = fields. String(required=True) sex = fields.String(required=True,error_messages={<!-- -->"required":"Sorry, permission must be filled"}) age = fields.Integer(missing=18,validate=validate.Range(min=18,max=40,error="Age must be between 18-40!")) # Limit the value range email = fields.Email(error_messages={<!-- -->"invalid":"Sorry, you must fill in the email format!"}) mobile = fields.String(required=True, validate=validate.Regexp("^1[3-9]\d{9}$",error="The mobile phone number format is incorrect"),error_messages= {<!-- -->"Regexp":"The mobile phone format is incorrect"}) @post_load def make_user_obj(self, data, **kwargs): return User(**data) def index3(): user_data = {<!-- -->"mobile":"1331345635","name": "xiaoming","age":40, "email": " [email protected]","sex":"abc"} us2 = UserSchema3() result = us2. load(user_data) result2 = us2. dumps(result) print(result) print(result2) return "ok" Custom authentication method from marshmallow import Schema, fields, validate, validates, ValidationError, post_load, validates_schema class UserSchema4(Schema): name = fields. String(required=True) sex = fields.String(required=True,error_messages={<!-- -->"required":"Sorry, permission must be filled"}) age = fields.Integer(missing=18,validate=validate.Range(min=18,max=40,error="Age must be between 18-40!")) # Limit the value range email = fields.Email(error_messages={<!-- -->"invalid":"Sorry, you must fill in the email format!"}) mobile = fields.String(required=True, validate=validate.Regexp("^1[3-9]\d{9}$",error="The mobile phone number format is incorrect"),error_messages= {<!-- -->"Regexp":"The mobile phone format is incorrect"}) password = fields. String(required=True, load_only=True) password2 = fields. String(required=True, allow_none=True) @post_load def make_user_obj(self, data, **kwargs): return User(**data) @validates("name") def validate_name(self, data, **kwargs): print("name=%s" % data) if data == "root": raise ValidationError({<!-- -->"Sorry, the root user is a super user! You do not have permission to register!"}) # must have a return value return data @validates_schema def validate(self, data, **kwargs): print(data) if data["password"] != data["password2"]: raise ValidationError("The password and confirmation password must be the same!") data. pop("password2") return data def index(): user_data = {<!-- -->"password":"12345","password2":"123456","mobile":"13313345635","name\ ": "root1","age":40, "email": "[email protected]","sex":"abc"} us2 = UserSchema4() result = us2. load(user_data) print(result) return "ok" Model constructor (ModelSchema) Official documentation: https://github.com/marshmallow-code/marshmallow-sqlalchemy ? https://marshmallow-sqlalchemy.readthedocs.io/en/latest/ Note: flask_marshmallow has removed the two model constructor classes, ModelSchema and TableSchema, after version 0.12.0, and officially recommends the use of SQLAlchemyAutoSchema and SQLAlchemySchema. The usage of the two classes is similar. create class UserSchema(SQLAlchemyAutoSchema): classMeta: model = model class name # table = models.Album.__table__ include_relationships = True # Whether to process foreign keys at the same time when outputting model objects include_fk = True # Whether the sequence stage also returns the primary key load_instance = True # In the deserialization phase, directly return the model object sql_session = db.session # database connection session object # fields= ["id","name"] # Started field list exclude = ["id","name"] # list of excluded fields ModelSchema Official documentation: https://github.com/marshmallow-code/marshmallow-sqlalchemy ? https://marshmallow-sqlalchemy.readthedocs.io/en/latest/ Note: flask_marshmallow has removed the two model constructor classes, ModelSchema and TableSchema, after version 0.12.0, and officially recommends the use of SQLAlchemyAutoSchema and SQLAlchemySchema. The usage of the two classes is similar. code: from marshmallow_sqlalchemy import SQLAlchemySchema,SQLAlchemyAutoSchema,auto_field from marshmallow import Schema, fields from application.apps.user.models import User, UserProfile, db from marshmallow import Schema, fields, validate, ValidationError, post_load class UserSchema5(SQLAlchemySchema): # The role of auto_field, set the type and option declaration of the current data field to automatically extract from the corresponding field in the model # name = auto_field() # Here, there is no username in the database at all, you need to declare the type of the current data dictionary and the field of the model from which the option declaration is extracted in the first parameter position username = auto_field("name",dump_only=True) # You can add or overwrite the original statement in the model on the basis of the original field created_time = auto_field(format="%Y-%m-%d") # format set time format # You can even declare some fields that are not models token = fields. String() classMeta: model = User fields = ["username","created_time","token"] def index5(): """Serialization processing of single model data""" from datetime import datetime user1 = User( name="xiaoming", password="123456", age=16, email="[email protected]", money=31.50, created_time = datetime.now(), ) user1.token = "abc" # Convert the model object to dictionary format data1 = UserSchema5().dump(user1) print(type(data1),data1) # <class 'dict'> {'username': 'xiaoming', 'token': 'abc', 'created_time': '2020-12-02'} return "ok" """SQLAlchemySchema is much simpler to use than the above Schema, but it is still necessary to write all the small conversion fields uniformly before converting these fields , if you don't want to write field information and copy directly from the model, you can also use SQLAlchemyAutoSchema. """ class UserSchema6(SQLAlchemyAutoSchema): token = fields. String() classMeta: model = User include_fk = False # enable foreign key relationship include_relationships = False # model relationship external attributes fields = ["name","created_time","info","token"] # If you want to replace all fields, just don't declare fields or exclude fields sql_session = db.session def index(): """Serialization processing of single model data""" from datetime import datetime user1 = User( name="xiaoming", password="123456", age=16, email="[email protected]", money=31.50, created_time = datetime.now(), info=UserProfile(position="teaching assistant") ) # Convert the model object to dictionary format user1.token="abcccccc" data1 = UserSchema6().dump(user1) print(type(data1),data1) # class 'dict'> {'info': <UserProfile: None>, 'created_time': '2020-12-02T17:47:21.821940', 'token': 'abcccccc ', 'name': 'xiaoming'} return "ok" Post navigation Coupons use a rules engine to calculate offers (drools)This deep analysis of the execution process of SpringMVC will help you improve to a higher level immediately after reading it!