Table of Contents
Overview
Main issues to consider
The description of this code is as follows
code show as below
usage
Overview
This is the metadata protocol I use for shared memory data identification. Neither shared memory nor network protocols should assume that the other party is the same as themselves – different programming languages, different hardware architectures, and data should be defined at the byte level.
Main issues to consider
- Endianness, the hardware endianness of different architectures is different. If you only run it under the same architecture, you can ignore it. However, once it is sent to other architectures, it will be too late to modify the program, so it is best to consider it from the beginning.
- Data type length. The length of data types is undefined in C and C++ and can have different alignments, so treating the entire structure (struct) as a data block is not feasible.
- Data alignment during operation. Data read from the network or file may not necessarily meet the address alignment requirements. Forced type conversion may cause the program to exit due to address alignment errors.
- Character encoding, I haven’t thought about this. When we write programs, we generally only recognize integers and ASCII characters. For Chinese characters, we have to figure it out on the front end.
The description of this code is as follows
Metadata protocols specify how metainformation about shared data structures is expressed.
For data such as shared memory, it is necessary to check the data format before accessing the data.
Changes in the definition of shared data structures may result in incorrect access to shared data
Differences in alignment at compile time can also result in incorrect access to shared data.
Therefore, the data element information specified in bytes is placed in the header of the shared data.
Since metainformation is defined in bytes, there is no format confusion problem
Meta information is defined as follows:
All bytes are signed
The entire meta information size is 256 bytes
Starting from offset 0, the order is:
0 GUID 36 bytes Contains a GUID string, registration table format, such as “A880D4F7-20AC-49BC-A58F-705D3BF05230”
means a unique identification of data
36 SIZEOFSHORT 1 byte Number of bytes of short
37 SIZEOFINT 1 byte Number of bytes of int
38 SIZEOFLONG 1 byte long number of bytes
39 BYTEORDER 1 byte Host byte order, the low bit is placed at the low address and is 1, otherwise it is 0
40 + n*4 USERINT array, each 4 bytes user-defined integer, host byte order, 54 in total, if not used, it should be set to 0
The code is as follows
//Metadata auxiliary class classCMeta { signed char data[256]; //Don't try to use int to pretend to be 4 bytes public: bool Set(signed char const * szGuid, int const * pInt, int nIntCount) { if(sizeof(int)!=4) { cout<<"fatal error : sizeof(int)!=4 , class Meta can not work . "<<endl; return false; } if(NULL==szGuid || NULL==pInt || strlen((char const *)szGuid)>36 || nIntCount>54)return false; memset(data,0,256); strcpy((char *)(data + 0),(char const *)szGuid); data[36]=(signed char)sizeof(short); data[37]=(signed char)sizeof(int); data[38]=(signed char)sizeof(long); int tempint=1; data[39]=(signed char)*(signed char *) & amp;tempint; memcpy(data + 40,pInt,4L*nIntCount); return true; } string & amp; toString(string & amp; s)const { s=""; if(sizeof(int)!=4) { s="fatal error : sizeof(int)!=4 , class Meta can not work . \\ "; return s; } char buf[1024]; memcpy(buf,data,36); buf[36]='\0'; s + ="GUID = "; s + =buf; s + =" \\ "; sprintf(buf,"sizeof short= %d int= %d long= %d byteorder= %d\\ ",data[36],data[37],data[38],data[39]); s + =buf; int i,k; k=0; for(i=0;i<54; + + i) { int temp; memcpy( & temp,data + 40 + 4*i,4);//Note that due to alignment issues, direct conversion to int* may cause core dump! So copy it once like this if(0!=temp)k=i;//Find the largest i } for(i=0;i<=k; + + i) { int temp; memcpy( & temp,data + 40 + 4*i,4);//Note that due to alignment issues, direct conversion to int* may cause core dump! So copy it once like this sprintf(buf,"- = m ",i,temp); s + =buf; if(0==(i + 1)%6)s + ="\\ "; } return s; } bool Compare(CMeta const & amp; tmp, string & amp; msg)const { if (0 != memcmp(data, tmp.data, 256)) { stringstream ss; for (long i = 0; i < 256; + + i) { if (data[i] != tmp.data[i]) { ss << "byte" << i << " " << (int)(data[i]) << " " << (int)(tmp.data[i]) << endl; } } msg = ss.str(); return false; } else { return true; } } bool CheckGuid(signed char const * szGuid)const { return 0 == memcmp(data, szGuid, 36); } bool CheckSys()const { if(data[36]!=(signed char)sizeof(short))return false; if(data[37]!=(signed char)sizeof(int))return false; if(data[38]!=(signed char)sizeof(long))return false; int tempint=1; if(data[39]!=(signed char)*(signed char *) & amp;tempint)return false; return true; } int GetInt(int i)const { int ret; memcpy( & amp;ret, data + 40 + i * sizeof(int), sizeof(int)); return ret; } int GetIntCount()const { int i, k; k = -1; for (i = 0; i < 54; + + i) { int temp; memcpy( & temp, data + 40 + 4 * i, 4);//Note that due to alignment issues, direct conversion to int* may cause core dump! So copy it once like this if (0 != temp)k = i;//Find the largest i } return k + 1; } bool CopyTo(signed char * pTo)const { memcpy(pTo,data,256); return true; } bool CopyFrom(signed char const * pFrom) { memcpy(data,pFrom,256); return true; } };
Usage
Set is filled with a GUID and a set of user-defined integers, and values such as endianness are automatically set. Generally the size and version of the incoming data structure
Compare Compare with another
CheckGuid compares whether the GUID is consistent. In my usage method, the GUID is the same but other differences indicate that different versions or models need to be processed. Inconsistent GUIDs indicate that the data block is not the expected one.
CheckSys checks whether the endianness and integer size are the same. This method is poorly named.
GetInt gets an integer value
GetIntCount gets the number of set integers. It doesn’t mean much. If the last integer is 0, it will be ignored, so it cannot be used to get how many integers were originally set.
CopyTo copies to the memory area, which is actually memcpy.
CopyFrom copies from the memory area, which is actually memcpy.
Generally, after reading data from the network or file, you should first verify whether the metadata meets expectations. If it does not meet expectations, you should not process it randomly.
Memory structs and network and file data generally need to be converted. Unless the data structure is carefully planned to ensure that it conforms to the hardware architecture and alignment settings, it is difficult to guarantee during network transmission, but it is easier for a single architecture.
However, even with a single architecture, there is still the problem of data structure versioning, so correct metadata processing is still necessary.
(Here is the end)