Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for serialised data in raccoon client #100

Open
punit-kulal opened this issue Oct 4, 2024 · 0 comments
Open

Support for serialised data in raccoon client #100

punit-kulal opened this issue Oct 4, 2024 · 0 comments

Comments

@punit-kulal
Copy link
Contributor

punit-kulal commented Oct 4, 2024

Summary
Currently, the raccoon, client supports in built serialization for either PROTO(ProtoMessage Object), or JSON. This means for already serialised data like in case for users of Stencil, they have to explicitly parse the message into a DynamicMessage for it to be of serializer Proto. We need to add support for a Serializer.RAW which forwards the message as it is without explicitly parsing the message they have serialised.

Proposed solution
Introduce Serialiser.RAW which gives the input as it is after checking for type safety

// SerializerFunc defines a conversion for raccoon message to byte sequence.
type SerializerFunc func(interface{}) ([]byte, error)

var (
	// json raccoon message serializer
	JSON = json.Marshal

	// proto raccoon message serializer
	PROTO = func(m interface{}) ([]byte, error) {
		msg, ok := m.(proto.Message)
		if !ok {
			return nil, errors.New("unable to marshal non proto")
		}
		return proto.Marshal(msg)
	}

      // New serialiser type proposed.
       RAW = func(m interface{}) ([]byte, error) {
	        typedM, ok := m.([]byte)
	        if !ok {
		        return nil, errors.New("failed to convert to byte array")
	        }
	       return typedM, nil
       } 
	
)

Additional context
Currently for stencil users,

        stencilUrl := fmt.Sprintf(stencilSchemaTemplate, namespace, schemaName)
	client, err := stencil.NewClient([]string{stencilUrl}, stencil.Options{})
	if err != nil {
		return nil, err
	}
	serialisedData, err := client.Serialize(packageName, data) // is of type []byte
	protoMessage, err := client.Parse(packageName, serialisedData) // additional step for converting the data to a protoMessage.
	if err != nil {
		return nil, err
	}
	return protoMessage, err

instead of directly returning the serialisedData.

@punit-kulal punit-kulal changed the title Support for serialised data during ingestion Support for serialised data in raccoon client Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant