<div align="center">
<img width="65%" alt="" src="docs/images/logo/fury_github_banner.png"><br>
</div>
[](https://github.com/alipay/fury)
[](https://twitter.com/fury_community)
Fury is a blazing-fast multi-language serialization framework powered by **jit(just-in-time compilation)** and **zero-copy**, providing up to 170x performance and ultimate ease of use.
https://furyio.org
## Features
- **Multiple languages**: Java/Python/C++/Golang/JavaScript/Rust/Scala/TypeScript.
- **Zero-copy**: Cross-language out-of-band serialization inspired
by [pickle5](https://peps.python.org/pep-0574/) and off-heap read/write.
- **High performance**: A highly-extensible JIT framework to generate serializer code at runtime in an async multi-thread way to speed serialization, providing 20-170x speed up by:
- reduce memory access by inlining variables in generated code.
- reduce virtual method invocation by inline call in generated code.
- reduce conditional branching.
- reduce hash lookup.
- **Multiple binary protocols**: Object graph, row format, and so on.
In addition to cross-language serialization, Fury also features at:
- Drop-in replace Java serialization frameworks such as JDK/Kryo/Hessian without modifying any code, but 100x faster.
It can greatly improve the efficiency of high-performance RPC calls, data transfer, and object persistence.
- **100% compatible** with JDK serialization API with much faster implementation: supporting JDK `writeObject/readObject/writeReplace/readResolve/readObjectNoData/Externalizable` API.
- Supports **Java 8~21**, Java 17+ `record` is supported too.
- Support [AOT compilation serialization](docs/guide/graalvm_guide.md) for **GraalVM native image**, and no reflection/serialization json config are needed.
- Supports shared and circular reference object serialization for golang.
- Supports [scala serialization](docs/guide/scala_guide.md)
- Supports automatic object serialization for golang.
## Protocols
Different scenarios have different serialization requirements. Fury designed and implemented
multiple binary protocols for those requirements:
- **Cross-language object graph protocol**:
- Cross-language serialize any object automatically, no need for IDL definition, schema compilation and object to/from protocol
conversion.
- Support shared reference and circular reference, no duplicate data or recursion error.
- Support object polymorphism.
- **Native java/python object graph protocol**: Highly-optimized based on type system of the language.
- **Row format protocol**: A cache-friendly binary random access format, supports skipping serialization and partial serialization,
and can convert to column-format automatically.
New protocols can be easily added based on fury existing buffer, encoding, meta, codegen and other capabilities. All of those share the same codebase, and the optimization for one protocol
can be reused by another protocol.
## Benchmarks
Different serialization frameworks are suitable for different scenarios, and benchmark results here are for reference only.
If you need to benchmark for your specific scenario, make sure all serialization frameworks are appropriately configured for that scenario.
Dynamic serialization frameworks support polymorphism and references, but they often come with a higher cost compared to static serialization frameworks, unless they utilize JIT techniques like Fury does.
Because Fury generates code at runtime, it is recommended to **warm up** the system before collecting benchmark statistics.
### Java Serialization
Title containing "compatible" represent schema compatible mode: support type forward/backward compatibility.
Title without "compatible" represent schema consistent mode: class schema must be the same between serialization and deserialization.
`Struct` is a class with [100 primitive fields](https://github.com/alipay/fury/tree/main/docs/benchmarks#Struct), `MediaContent` is a class from [jvm-serializers](https://github.com/eishay/jvm-serializers/blob/master/tpc/src/data/media/MediaContent.java), `Sample` is a class from [kryo benchmark](https://github.com/EsotericSoftware/kryo/blob/master/benchmarks/src/main/java/com/esotericsoftware/kryo/benchmarks/data/Sample.java).
<p align="center">
<img width="24%" alt="" src="docs/benchmarks/compatible/bench_serialize_compatible_STRUCT_to_directBuffer_tps.png">
<img width="24%" alt="" src="docs/benchmarks/compatible/bench_serialize_compatible_MEDIA_CONTENT_to_array_tps.png">
<img width="24%" alt="" src="docs/benchmarks/serialization/bench_serialize_MEDIA_CONTENT_to_array_tps.png">
<img width="24%" alt="" src="docs/benchmarks/serialization/bench_serialize_SAMPLE_to_array_tps.png">
</p>
<p align="center">
<img width="24%" alt="" src="docs/benchmarks/compatible/bench_deserialize_compatible_STRUCT_from_directBuffer_tps.png">
<img width="24%" alt="" src="docs/benchmarks/compatible/bench_deserialize_compatible_MEDIA_CONTENT_from_array_tps.png">
<img width="24%" alt="" src="docs/benchmarks/deserialization/bench_deserialize_MEDIA_CONTENT_from_array_tps.png">
<img width="24%" alt="" src="docs/benchmarks/deserialization/bench_deserialize_SAMPLE_from_array_tps.png">
</p>
See [benchmarks](https://github.com/alipay/fury/tree/main/docs/benchmarks) for more benchmarks about type forward/backward compatibility, off-heap support, zero-copy serialization.
## Installation
### Java
Nightly snapshot:
```xml
<repositories>
<repository>
<id>sonatype</id>
<url>https://s01.oss.sonatype.org/content/repositories/snapshots</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
<dependency>
<groupId>org.furyio</groupId>
<artifactId>fury-core</artifactId>
<version>0.5.0-SNAPSHOT</version>
</dependency>
<!-- row/arrow format support -->
<!-- <dependency>
<groupId>org.furyio</groupId>
<artifactId>fury-format</artifactId>
<version>0.5.0-SNAPSHOT</version>
</dependency> -->
```
Release version:
```xml
<dependency>
<groupId>org.furyio</groupId>
<artifactId>fury-core</artifactId>
<version>0.4.1</version>
</dependency>
<!-- row/arrow format support -->
<!-- <dependency>
<groupId>org.furyio</groupId>
<artifactId>fury-format</artifactId>
<version>0.4.1</version>
</dependency> -->
```
### Scala
```sbt
libraryDependencies += "org.furyio" % "fury-core" % "0.4.1"
```
### Python
```bash
# Release version will be provided in the future.
pip install pyfury --pre
```
### JavaScript
```bash
npm install @furyjs/fury
```
### Golang
```bash
go get github.com/alipay/fury/go/fury
```
## Quickstart
Here we give a quick start about how to use fury, see [user guide](https://github.com/alipay/fury/blob/main/docs/README.md) for more details about [java](https://github.com/alipay/fury/blob/main/docs/guide/java_object_graph_guide.md), [cross language](https://github.com/alipay/fury/blob/main/docs/guide/xlang_object_graph_guide.md), and [row format](https://github.com/alipay/fury/blob/main/docs/guide/row_format_guide.md).
### Fury java object graph serialization
If you don't have cross-language requirements, using this mode will
have better performance.
```java
import io.fury.*;
import io.fury.config.*;
import java.util.*;
public class Example {
public static void main(String[] args) {
SomeClass object = new SomeClass();
// Note that Fury instances should be reused between
// multiple serializations of different objects.
{
Fury fury = Fury.builder().withLanguage(Language.JAVA)
// Allow to deserialize objects unknown types, more flexible
// but may be insecure if the classes contains malicious code.
.requireClassRegistration(false)
.build();
// Registering types can reduce class name serialization overhead, but not mandatory.
// If class registration enabled, all custom types must be registered.
fury.register(SomeClass.class);
byte[] bytes = fury.serialize(object);
System.out.println(fury.deserialize(bytes));
}
{
ThreadSafeFury fury = Fury.builder().withLanguage(Language.JAVA)
// Allow to deserialize objects unknown types, more flexible
// but may be insecure if the classes contains malicious code.
.requireClassRegistration(false)
.buildThreadSafeFury();
byte[] bytes = fury.serialize(object);
System.out.println(fury.deserialize(bytes));
}
{
ThreadSafeFury fury = new ThreadLocalFury(classLoader -> {
Fury f = Fury.builder().withLanguage(Language.JAVA)
.withClassLoader(classLoader).build();
f.register(SomeClass.class);
return f;
});
byte[] bytes = fury.serialize(object);
System.out.println(fury.deserialize(bytes));
}
}
}
```
### Cross-language object graph serialization
**Java**
```java
import io.fury.*;
import io.fury.config.*;
import java.util.*;
public class ReferenceExample {
public static class SomeClass {
SomeClass f1;
Map<String, String> f2;
Map<String, String> f3;
}
public static Object createObject() {
SomeClass obj = new SomeClass();
obj.f1 = obj;
obj.f2 = ofHashMap("k1", "v1", "k2", "v2");
obj.f3 = obj.f2;
return obj;
}
// mvn exec:java -Dexec.mainClass="io.fury.examples.ReferenceExample"
public static void main(String[] args) {
Fury fury = Fury.builder().withLanguage(Language.XLANG)
.withRefTracking(true).build();
fury.register(SomeClass.class, "example.SomeClass");
byte[] bytes = fury.serialize(createObject());
// bytes can be data serialized by other languages.
System.out.println(fury.deserialize(bytes));
}
}
```
**Python**
```python
from typing import Dict
import pyfury
class SomeClass:
f1: "SomeClass"
f2: Dict[str, str]
f3: Dict[str, str]
fury = pyfury.Fury(ref_tracking=True)
fury.register_class(SomeClass, type_tag="example.SomeClass")
obj = SomeClass()
obj.f2 = {"k1": "v1", "k2": "v2"}
obj.f1, obj.f3 = obj, obj.f2
data = fury.serialize(obj)
# bytes can be data serialized by other languages.
print(fury.deserialize(data))
```
**Golang**
```go
package main
import furygo "github.com/alipay/fury/go/fury"
import "fmt"
func main() {
type SomeClass struct {
F1 *SomeClass
F2 map[string]string
F3 map[string]string
}
fury := furygo.NewFury(true)
if err := fury.RegisterTagType("example.SomeClass", SomeClass{}); err != nil {
panic(err)
}
value := &SomeClass{F2: map[string]string{"k1": "v1", "k2": "v2"}}
value.F3 = value.F2
value.F1 = value
bytes, err := fury.Marshal(value)
if err != nil {
}
var newValue interface{}
// bytes can be data serialized by other languages.
if err := fury.Unmarshal(bytes, &newValue); err != nil {
panic(err)
}
fmt.Println(newValue)
}
```
### Row format
#### Java
```java
public class Bar {
String f1;
List<Long> f2;
}
public class Foo {
int f1;
List<Integer> f2;
Map<String, Integer> f3;
List<Bar> f4;
}
RowEncoder<Foo> encoder = Encoders.bean(Foo.class);
Foo foo = new Foo();
foo.f1 = 10;
foo.f2 = IntStream.range(0, 1000000).boxed().collect(Collectors.toList());
foo.f3 = IntStream.range(0, 1000000).boxed().collect(Collectors.toMap(i -> "k"+i, i->i));
List<Bar> bars = new ArrayList<>(1000000);
for (int i = 0; i < 1000000; i++) {
Bar bar = new Bar();
bar.f1 = "s"+i;
bar.f2 = LongStream.range(0, 10).boxed().collect(Collectors.toList());
bars.add(bar);
}
foo.f4 = bars;
// Can be zero-copy read by python
BinaryRow binaryRow = encoder.toRow(foo);
// can be data from python
Foo newFoo = encoder.fromRow(binaryRow);
// zero-copy read List<Integer> f2
BinaryArray binaryArray2 = binaryRow.getArray(1);
// zero-copy read List<Bar> f4
BinaryArray binaryArray4 = binaryRow.getArray(3);
// zero-copy read 11th element of `readList<Bar> f4`
BinaryRow barStruct = binaryArray4.getStruct(10);
// zero-copy read 6th of f2 of 11th element of `readList<Bar> f4`
barStruct.getArray(1).getLong(5);
RowEncoder<Bar> barEncoder = Encoders.bean(Bar.class);
// deserialize part of data.
Bar newBar = barEncoder.fromRow(barStruct);
Bar newBar2 = barEncoder.fromRow(binaryArray4.getStruct(20));
```
#### Python
```python
@dataclass
class Bar:
f1: str
f2: List[pa.int64]
@dataclass
class Foo:
f1: pa.int32
f2: List[pa.int32]
f3: Dict[str, pa.int32]
f4: List[Bar]
encoder = pyfury.encoder(Foo)
foo = Foo(f1=10, f2=list(range(1000_000)),
f3={f"k{i}": i for i in range(1000_000)},
f4=[Bar(f1=f"s{i}", f2=list(range(10))) for i in range(1000_000)])
binary: bytes = encoder.to_row(foo).to_bytes()
foo_row = pyfury.RowData(encoder.schema, binary)
print(foo_row.f2[100000], foo_row.f4[100000].f1, foo_row.f4[200000].f2[5])
```
## Compatibility
### Schema Compatibility
Fury java object graph serialization support class schema forward/backward compatibility. The serialization peer and deserialization peer can add/delete fields independently.
We plan to add support cross-language serialization after [meta compression](https://github.com/alipay/fury/issues/203) is finished.
### Binary Compatibility
We are still improving our protocols, binary compatibility is not ensured between fury major releases for now.
it's ensured between minor versions only. Please
`versioning` your data by fury major version if you will upgrade fury in the future, see [how to upgrade fury](https://github.com/alipay/fury/blob/main/docs/guide/java_object_graph_guide.md#upgrade-fury) for further details.
Binary compatibility will be ensured when fury 1.0 is released.
## Security
Static serialization is secure. But dynamic serialization such as fury java/python native serialization supports deserializing unregistered types, which provides more dynamics and flexibility, but also introduce security risks.
For example, the deserialization may invoke `init` constructor or `equals`/`hashCode` method, if the method body contains malicious code, the system will be at risk.
Fury provides a class registration option that is enabled by default for such protocols, allowing only deserialization of trusted registered types or built-in types.
**Do not disable class registration unless you can ensure your environment is secure**.
If this option is disabled, you are responsible for serialization security. You can configure `io.fury.resolver.ClassChecker` by
`ClassResolver#setClassChecker` to control which classes are allowed for serialization.
## How to Build
Please read the [BUILD](docs/guide/DEVELOPMENT.md) guide for instructions on how to build.
## How to Contribute
Please read the [CONTRIBUTING](CONTRIBUTING.md) guide for instructions on how to contribute.
For ecosystem projects, please see https://github.com/fury-project
## License
Licensed under the [Apache License, Version 2.0](LICENSE)
Raw data
{
"_id": null,
"home_page": "",
"name": "pyfury",
"maintainer": "https://github.com/chaokunyang",
"docs_url": null,
"requires_python": "",
"maintainer_email": "shawn.ck.yang@gmail.com",
"keywords": "fury serialization multi-language arrow row-format jit vectorization zero-copy",
"author": "chaokunyang",
"author_email": "shawn.ck.yang@gmail.com",
"download_url": "",
"platform": null,
"description": "<div align=\"center\">\n <img width=\"65%\" alt=\"\" src=\"docs/images/logo/fury_github_banner.png\"><br>\n</div>\n\n[](https://github.com/alipay/fury)\n[](https://twitter.com/fury_community)\n\n\nFury is a blazing-fast multi-language serialization framework powered by **jit(just-in-time compilation)** and **zero-copy**, providing up to 170x performance and ultimate ease of use.\n\nhttps://furyio.org\n\n## Features\n- **Multiple languages**: Java/Python/C++/Golang/JavaScript/Rust/Scala/TypeScript.\n- **Zero-copy**: Cross-language out-of-band serialization inspired\n by [pickle5](https://peps.python.org/pep-0574/) and off-heap read/write.\n- **High performance**: A highly-extensible JIT framework to generate serializer code at runtime in an async multi-thread way to speed serialization, providing 20-170x speed up by:\n - reduce memory access by inlining variables in generated code.\n - reduce virtual method invocation by inline call in generated code.\n - reduce conditional branching.\n - reduce hash lookup.\n- **Multiple binary protocols**: Object graph, row format, and so on.\n\nIn addition to cross-language serialization, Fury also features at:\n\n- Drop-in replace Java serialization frameworks such as JDK/Kryo/Hessian without modifying any code, but 100x faster. \n It can greatly improve the efficiency of high-performance RPC calls, data transfer, and object persistence.\n- **100% compatible** with JDK serialization API with much faster implementation: supporting JDK `writeObject/readObject/writeReplace/readResolve/readObjectNoData/Externalizable` API. \n- Supports **Java 8~21**, Java 17+ `record` is supported too.\n- Support [AOT compilation serialization](docs/guide/graalvm_guide.md) for **GraalVM native image**, and no reflection/serialization json config are needed.\n- Supports shared and circular reference object serialization for golang.\n- Supports [scala serialization](docs/guide/scala_guide.md)\n- Supports automatic object serialization for golang.\n\n## Protocols\nDifferent scenarios have different serialization requirements. Fury designed and implemented \nmultiple binary protocols for those requirements:\n- **Cross-language object graph protocol**:\n - Cross-language serialize any object automatically, no need for IDL definition, schema compilation and object to/from protocol\n conversion.\n - Support shared reference and circular reference, no duplicate data or recursion error.\n - Support object polymorphism.\n- **Native java/python object graph protocol**: Highly-optimized based on type system of the language.\n- **Row format protocol**: A cache-friendly binary random access format, supports skipping serialization and partial serialization,\n and can convert to column-format automatically.\n\nNew protocols can be easily added based on fury existing buffer, encoding, meta, codegen and other capabilities. All of those share the same codebase, and the optimization for one protocol\ncan be reused by another protocol.\n\n## Benchmarks\nDifferent serialization frameworks are suitable for different scenarios, and benchmark results here are for reference only.\n\nIf you need to benchmark for your specific scenario, make sure all serialization frameworks are appropriately configured for that scenario.\n\nDynamic serialization frameworks support polymorphism and references, but they often come with a higher cost compared to static serialization frameworks, unless they utilize JIT techniques like Fury does.\nBecause Fury generates code at runtime, it is recommended to **warm up** the system before collecting benchmark statistics.\n\n### Java Serialization\nTitle containing \"compatible\" represent schema compatible mode: support type forward/backward compatibility.\n\nTitle without \"compatible\" represent schema consistent mode: class schema must be the same between serialization and deserialization.\n\n`Struct` is a class with [100 primitive fields](https://github.com/alipay/fury/tree/main/docs/benchmarks#Struct), `MediaContent` is a class from [jvm-serializers](https://github.com/eishay/jvm-serializers/blob/master/tpc/src/data/media/MediaContent.java), `Sample` is a class from [kryo benchmark](https://github.com/EsotericSoftware/kryo/blob/master/benchmarks/src/main/java/com/esotericsoftware/kryo/benchmarks/data/Sample.java).\n\n<p align=\"center\">\n<img width=\"24%\" alt=\"\" src=\"docs/benchmarks/compatible/bench_serialize_compatible_STRUCT_to_directBuffer_tps.png\">\n<img width=\"24%\" alt=\"\" src=\"docs/benchmarks/compatible/bench_serialize_compatible_MEDIA_CONTENT_to_array_tps.png\">\n<img width=\"24%\" alt=\"\" src=\"docs/benchmarks/serialization/bench_serialize_MEDIA_CONTENT_to_array_tps.png\">\n<img width=\"24%\" alt=\"\" src=\"docs/benchmarks/serialization/bench_serialize_SAMPLE_to_array_tps.png\">\n</p>\n\n<p align=\"center\">\n<img width=\"24%\" alt=\"\" src=\"docs/benchmarks/compatible/bench_deserialize_compatible_STRUCT_from_directBuffer_tps.png\">\n<img width=\"24%\" alt=\"\" src=\"docs/benchmarks/compatible/bench_deserialize_compatible_MEDIA_CONTENT_from_array_tps.png\">\n<img width=\"24%\" alt=\"\" src=\"docs/benchmarks/deserialization/bench_deserialize_MEDIA_CONTENT_from_array_tps.png\">\n<img width=\"24%\" alt=\"\" src=\"docs/benchmarks/deserialization/bench_deserialize_SAMPLE_from_array_tps.png\">\n</p>\n\nSee [benchmarks](https://github.com/alipay/fury/tree/main/docs/benchmarks) for more benchmarks about type forward/backward compatibility, off-heap support, zero-copy serialization.\n\n## Installation\n### Java\nNightly snapshot:\n```xml\n<repositories>\n <repository>\n <id>sonatype</id>\n <url>https://s01.oss.sonatype.org/content/repositories/snapshots</url>\n <releases>\n <enabled>false</enabled>\n </releases>\n <snapshots>\n <enabled>true</enabled>\n </snapshots>\n </repository>\n</repositories>\n<dependency>\n <groupId>org.furyio</groupId>\n <artifactId>fury-core</artifactId>\n <version>0.5.0-SNAPSHOT</version>\n</dependency>\n<!-- row/arrow format support -->\n<!-- <dependency>\n <groupId>org.furyio</groupId>\n <artifactId>fury-format</artifactId>\n <version>0.5.0-SNAPSHOT</version>\n</dependency> -->\n```\nRelease version:\n```xml\n<dependency>\n <groupId>org.furyio</groupId>\n <artifactId>fury-core</artifactId>\n <version>0.4.1</version>\n</dependency>\n<!-- row/arrow format support -->\n<!-- <dependency>\n <groupId>org.furyio</groupId>\n <artifactId>fury-format</artifactId>\n <version>0.4.1</version>\n</dependency> -->\n```\n\n### Scala\n```sbt\nlibraryDependencies += \"org.furyio\" % \"fury-core\" % \"0.4.1\"\n```\n\n### Python\n```bash\n# Release version will be provided in the future.\npip install pyfury --pre\n```\n\n### JavaScript\n```bash\nnpm install @furyjs/fury\n```\n\n### Golang\n```bash\ngo get github.com/alipay/fury/go/fury\n```\n\n## Quickstart\nHere we give a quick start about how to use fury, see [user guide](https://github.com/alipay/fury/blob/main/docs/README.md) for more details about [java](https://github.com/alipay/fury/blob/main/docs/guide/java_object_graph_guide.md), [cross language](https://github.com/alipay/fury/blob/main/docs/guide/xlang_object_graph_guide.md), and [row format](https://github.com/alipay/fury/blob/main/docs/guide/row_format_guide.md).\n\n### Fury java object graph serialization\nIf you don't have cross-language requirements, using this mode will \nhave better performance.\n```java\nimport io.fury.*;\nimport io.fury.config.*;\nimport java.util.*;\n\npublic class Example {\n public static void main(String[] args) {\n SomeClass object = new SomeClass();\n // Note that Fury instances should be reused between \n // multiple serializations of different objects.\n {\n Fury fury = Fury.builder().withLanguage(Language.JAVA)\n // Allow to deserialize objects unknown types, more flexible \n // but may be insecure if the classes contains malicious code.\n .requireClassRegistration(false)\n .build();\n // Registering types can reduce class name serialization overhead, but not mandatory.\n // If class registration enabled, all custom types must be registered.\n fury.register(SomeClass.class);\n byte[] bytes = fury.serialize(object);\n System.out.println(fury.deserialize(bytes));\n }\n {\n ThreadSafeFury fury = Fury.builder().withLanguage(Language.JAVA)\n // Allow to deserialize objects unknown types, more flexible \n // but may be insecure if the classes contains malicious code.\n .requireClassRegistration(false)\n .buildThreadSafeFury();\n byte[] bytes = fury.serialize(object);\n System.out.println(fury.deserialize(bytes));\n }\n {\n ThreadSafeFury fury = new ThreadLocalFury(classLoader -> {\n Fury f = Fury.builder().withLanguage(Language.JAVA)\n .withClassLoader(classLoader).build();\n f.register(SomeClass.class);\n return f;\n });\n byte[] bytes = fury.serialize(object);\n System.out.println(fury.deserialize(bytes));\n }\n }\n}\n```\n\n### Cross-language object graph serialization\n**Java**\n```java\nimport io.fury.*;\nimport io.fury.config.*;\nimport java.util.*;\n\npublic class ReferenceExample {\n public static class SomeClass {\n SomeClass f1;\n Map<String, String> f2;\n Map<String, String> f3;\n }\n\n public static Object createObject() {\n SomeClass obj = new SomeClass();\n obj.f1 = obj;\n obj.f2 = ofHashMap(\"k1\", \"v1\", \"k2\", \"v2\");\n obj.f3 = obj.f2;\n return obj;\n }\n\n // mvn exec:java -Dexec.mainClass=\"io.fury.examples.ReferenceExample\"\n public static void main(String[] args) {\n Fury fury = Fury.builder().withLanguage(Language.XLANG)\n .withRefTracking(true).build();\n fury.register(SomeClass.class, \"example.SomeClass\");\n byte[] bytes = fury.serialize(createObject());\n // bytes can be data serialized by other languages.\n System.out.println(fury.deserialize(bytes));\n }\n}\n```\n\n**Python**\n\n```python\nfrom typing import Dict\nimport pyfury\n\nclass SomeClass:\n f1: \"SomeClass\"\n f2: Dict[str, str]\n f3: Dict[str, str]\n\nfury = pyfury.Fury(ref_tracking=True)\nfury.register_class(SomeClass, type_tag=\"example.SomeClass\")\nobj = SomeClass()\nobj.f2 = {\"k1\": \"v1\", \"k2\": \"v2\"}\nobj.f1, obj.f3 = obj, obj.f2\ndata = fury.serialize(obj)\n# bytes can be data serialized by other languages.\nprint(fury.deserialize(data))\n```\n\n**Golang** \n\n```go\npackage main\n\nimport furygo \"github.com/alipay/fury/go/fury\"\nimport \"fmt\"\n\nfunc main() {\n\ttype SomeClass struct {\n\t\tF1 *SomeClass\n\t\tF2 map[string]string\n\t\tF3 map[string]string\n\t}\n\tfury := furygo.NewFury(true)\n\tif err := fury.RegisterTagType(\"example.SomeClass\", SomeClass{}); err != nil {\n\t\tpanic(err)\n\t}\n\tvalue := &SomeClass{F2: map[string]string{\"k1\": \"v1\", \"k2\": \"v2\"}}\n\tvalue.F3 = value.F2\n\tvalue.F1 = value\n\tbytes, err := fury.Marshal(value)\n\tif err != nil {\n\t}\n\tvar newValue interface{}\n\t// bytes can be data serialized by other languages.\n\tif err := fury.Unmarshal(bytes, &newValue); err != nil {\n\t\tpanic(err)\n\t}\n\tfmt.Println(newValue)\n}\n```\n\n### Row format\n#### Java\n```java\npublic class Bar {\n String f1;\n List<Long> f2;\n}\n\npublic class Foo {\n int f1;\n List<Integer> f2;\n Map<String, Integer> f3;\n List<Bar> f4;\n}\n\nRowEncoder<Foo> encoder = Encoders.bean(Foo.class);\nFoo foo = new Foo();\nfoo.f1 = 10;\nfoo.f2 = IntStream.range(0, 1000000).boxed().collect(Collectors.toList());\nfoo.f3 = IntStream.range(0, 1000000).boxed().collect(Collectors.toMap(i -> \"k\"+i, i->i));\nList<Bar> bars = new ArrayList<>(1000000);\nfor (int i = 0; i < 1000000; i++) {\n Bar bar = new Bar();\n bar.f1 = \"s\"+i;\n bar.f2 = LongStream.range(0, 10).boxed().collect(Collectors.toList());\n bars.add(bar);\n}\nfoo.f4 = bars;\n// Can be zero-copy read by python\nBinaryRow binaryRow = encoder.toRow(foo);\n// can be data from python\nFoo newFoo = encoder.fromRow(binaryRow);\n// zero-copy read List<Integer> f2\nBinaryArray binaryArray2 = binaryRow.getArray(1);\n// zero-copy read List<Bar> f4\nBinaryArray binaryArray4 = binaryRow.getArray(3);\n// zero-copy read 11th element of `readList<Bar> f4`\nBinaryRow barStruct = binaryArray4.getStruct(10);\n\n// zero-copy read 6th of f2 of 11th element of `readList<Bar> f4`\nbarStruct.getArray(1).getLong(5);\nRowEncoder<Bar> barEncoder = Encoders.bean(Bar.class);\n// deserialize part of data.\nBar newBar = barEncoder.fromRow(barStruct);\nBar newBar2 = barEncoder.fromRow(binaryArray4.getStruct(20));\n```\n#### Python\n```python\n@dataclass\nclass Bar:\n f1: str\n f2: List[pa.int64]\n@dataclass\nclass Foo:\n f1: pa.int32\n f2: List[pa.int32]\n f3: Dict[str, pa.int32]\n f4: List[Bar]\n\nencoder = pyfury.encoder(Foo)\nfoo = Foo(f1=10, f2=list(range(1000_000)),\n f3={f\"k{i}\": i for i in range(1000_000)},\n f4=[Bar(f1=f\"s{i}\", f2=list(range(10))) for i in range(1000_000)])\nbinary: bytes = encoder.to_row(foo).to_bytes()\nfoo_row = pyfury.RowData(encoder.schema, binary)\nprint(foo_row.f2[100000], foo_row.f4[100000].f1, foo_row.f4[200000].f2[5])\n```\n\n## Compatibility\n### Schema Compatibility\nFury java object graph serialization support class schema forward/backward compatibility. The serialization peer and deserialization peer can add/delete fields independently.\n\nWe plan to add support cross-language serialization after [meta compression](https://github.com/alipay/fury/issues/203) is finished.\n\n### Binary Compatibility\nWe are still improving our protocols, binary compatibility is not ensured between fury major releases for now.\nit's ensured between minor versions only. Please\n`versioning` your data by fury major version if you will upgrade fury in the future, see [how to upgrade fury](https://github.com/alipay/fury/blob/main/docs/guide/java_object_graph_guide.md#upgrade-fury) for further details.\n\nBinary compatibility will be ensured when fury 1.0 is released.\n\n## Security\nStatic serialization is secure. But dynamic serialization such as fury java/python native serialization supports deserializing unregistered types, which provides more dynamics and flexibility, but also introduce security risks.\n\nFor example, the deserialization may invoke `init` constructor or `equals`/`hashCode` method, if the method body contains malicious code, the system will be at risk.\n\nFury provides a class registration option that is enabled by default for such protocols, allowing only deserialization of trusted registered types or built-in types.\n **Do not disable class registration unless you can ensure your environment is secure**.\n\nIf this option is disabled, you are responsible for serialization security. You can configure `io.fury.resolver.ClassChecker` by\n`ClassResolver#setClassChecker` to control which classes are allowed for serialization.\n\n## How to Build\n\nPlease read the [BUILD](docs/guide/DEVELOPMENT.md) guide for instructions on how to build.\n\n## How to Contribute\n\nPlease read the [CONTRIBUTING](CONTRIBUTING.md) guide for instructions on how to contribute.\n\nFor ecosystem projects, please see https://github.com/fury-project\n\n## License\n\nLicensed under the [Apache License, Version 2.0](LICENSE)\n",
"bugtrack_url": null,
"license": "",
"summary": "Fury is a blazing fast multi-language serialization framework powered by jit, vectorization and zero-copy",
"version": "0.4.1",
"project_urls": null,
"split_keywords": [
"fury",
"serialization",
"multi-language",
"arrow",
"row-format",
"jit",
"vectorization",
"zero-copy"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "008c5bd90651aba3e8d858c59b1a438e2c933958904696a04d12ce94a3276ea8",
"md5": "464d8677794cb9579469068ad568813e",
"sha256": "9288de26152d5841f06cd0fa2a7e273b71957c18e7dabf61a55b29fd10c8aa8f"
},
"downloads": -1,
"filename": "pyfury-0.4.1-cp310-cp310-macosx_12_0_x86_64.whl",
"has_sig": false,
"md5_digest": "464d8677794cb9579469068ad568813e",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": null,
"size": 971956,
"upload_time": "2023-12-08T17:59:05",
"upload_time_iso_8601": "2023-12-08T17:59:05.116627Z",
"url": "https://files.pythonhosted.org/packages/00/8c/5bd90651aba3e8d858c59b1a438e2c933958904696a04d12ce94a3276ea8/pyfury-0.4.1-cp310-cp310-macosx_12_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1c64759d6cbabe3dea9d30034ce3519b18794535320a7fa4646d032ee2a7d35b",
"md5": "e3a04a008cbf03ec074166be6b3c2df0",
"sha256": "c8133ac9fb73f356de3995cb14f0647d7070fbf1d8c114b42b0d7a3699eff793"
},
"downloads": -1,
"filename": "pyfury-0.4.1-cp310-cp310-manylinux1_x86_64.whl",
"has_sig": false,
"md5_digest": "e3a04a008cbf03ec074166be6b3c2df0",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": null,
"size": 1200550,
"upload_time": "2023-12-08T17:14:40",
"upload_time_iso_8601": "2023-12-08T17:14:40.516796Z",
"url": "https://files.pythonhosted.org/packages/1c/64/759d6cbabe3dea9d30034ce3519b18794535320a7fa4646d032ee2a7d35b/pyfury-0.4.1-cp310-cp310-manylinux1_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c691374a9dcd83b2ca3b4e1da2399ef321218bfc1b15b77da379a682d0ee10d5",
"md5": "4d2f7bd6d6d1853e10b69c05a2516c14",
"sha256": "1d3b1a1f34c89c17c6081f50dce8a22f42e6c26cd12f15d12629799eadedeb60"
},
"downloads": -1,
"filename": "pyfury-0.4.1-cp311-cp311-macosx_12_0_x86_64.whl",
"has_sig": false,
"md5_digest": "4d2f7bd6d6d1853e10b69c05a2516c14",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": null,
"size": 973700,
"upload_time": "2023-12-08T17:59:07",
"upload_time_iso_8601": "2023-12-08T17:59:07.539017Z",
"url": "https://files.pythonhosted.org/packages/c6/91/374a9dcd83b2ca3b4e1da2399ef321218bfc1b15b77da379a682d0ee10d5/pyfury-0.4.1-cp311-cp311-macosx_12_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "249769ee3cb055477e9e08ed162c13cc9362a0c8b6a0a311d70b274c16566641",
"md5": "7674f5da60edd520bdf9ba561217796b",
"sha256": "c605b214e003fb2cd9f7b4916d3c395cf4ef021760d9b30c4bd36e68cb43382e"
},
"downloads": -1,
"filename": "pyfury-0.4.1-cp311-cp311-manylinux1_x86_64.whl",
"has_sig": false,
"md5_digest": "7674f5da60edd520bdf9ba561217796b",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": null,
"size": 1200518,
"upload_time": "2023-12-08T17:14:26",
"upload_time_iso_8601": "2023-12-08T17:14:26.963962Z",
"url": "https://files.pythonhosted.org/packages/24/97/69ee3cb055477e9e08ed162c13cc9362a0c8b6a0a311d70b274c16566641/pyfury-0.4.1-cp311-cp311-manylinux1_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "610b71055c932023b04349171fc4aead9f547e47ed2192ecb690e8390409dae4",
"md5": "aaa1a74f13b0ed34397e0a6c071d59f1",
"sha256": "9d8783a032c7590b73f80c573a395e61a0117bee779159316d05a4d4a6ec9bbe"
},
"downloads": -1,
"filename": "pyfury-0.4.1-cp312-cp312-macosx_12_0_x86_64.whl",
"has_sig": false,
"md5_digest": "aaa1a74f13b0ed34397e0a6c071d59f1",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": null,
"size": 959224,
"upload_time": "2023-12-08T17:59:09",
"upload_time_iso_8601": "2023-12-08T17:59:09.807100Z",
"url": "https://files.pythonhosted.org/packages/61/0b/71055c932023b04349171fc4aead9f547e47ed2192ecb690e8390409dae4/pyfury-0.4.1-cp312-cp312-macosx_12_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7dfab7abbdcdf5c494baca476235ed2bf8363db1d70da120aed2538356c4a0e9",
"md5": "7c388d1163d8cdd8994ad98efa380bff",
"sha256": "54c4bf0dc2c5beef01a1eeb8dc07d2e1244c12e2fee803d64f07eea98c268927"
},
"downloads": -1,
"filename": "pyfury-0.4.1-cp312-cp312-manylinux1_x86_64.whl",
"has_sig": false,
"md5_digest": "7c388d1163d8cdd8994ad98efa380bff",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": null,
"size": 1162426,
"upload_time": "2023-12-08T17:14:35",
"upload_time_iso_8601": "2023-12-08T17:14:35.077353Z",
"url": "https://files.pythonhosted.org/packages/7d/fa/b7abbdcdf5c494baca476235ed2bf8363db1d70da120aed2538356c4a0e9/pyfury-0.4.1-cp312-cp312-manylinux1_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "657719eec0177a72928ea0495b76714d111795fb4ee1e29f969f7ba56d5cbb5b",
"md5": "e46c912cd3c2a56aace000a16a2f1ee2",
"sha256": "13c5489763aaa9559c5d76264d09d09892355aa7eb4a4a36c3dffd34abdbff0c"
},
"downloads": -1,
"filename": "pyfury-0.4.1-cp37-cp37m-macosx_12_0_x86_64.whl",
"has_sig": false,
"md5_digest": "e46c912cd3c2a56aace000a16a2f1ee2",
"packagetype": "bdist_wheel",
"python_version": "cp37",
"requires_python": null,
"size": 952194,
"upload_time": "2023-12-08T17:59:11",
"upload_time_iso_8601": "2023-12-08T17:59:11.848716Z",
"url": "https://files.pythonhosted.org/packages/65/77/19eec0177a72928ea0495b76714d111795fb4ee1e29f969f7ba56d5cbb5b/pyfury-0.4.1-cp37-cp37m-macosx_12_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "74b67581104cd0e36b2b07e82c2681f8d27bf97af64a8603ebcc51ecedf22cf8",
"md5": "a0d08037d38aef6ea469430137593a6a",
"sha256": "4d31de96ff99307ff9d3d7b245d1078cd25689cd483bb2e1a442c73999556740"
},
"downloads": -1,
"filename": "pyfury-0.4.1-cp37-cp37m-manylinux1_x86_64.whl",
"has_sig": false,
"md5_digest": "a0d08037d38aef6ea469430137593a6a",
"packagetype": "bdist_wheel",
"python_version": "cp37",
"requires_python": null,
"size": 1158428,
"upload_time": "2023-12-08T17:14:29",
"upload_time_iso_8601": "2023-12-08T17:14:29.953408Z",
"url": "https://files.pythonhosted.org/packages/74/b6/7581104cd0e36b2b07e82c2681f8d27bf97af64a8603ebcc51ecedf22cf8/pyfury-0.4.1-cp37-cp37m-manylinux1_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7a24e6d5188d1b1692cd1cff9bf786f7b63353e5dcb6f7690e78bb362327aed4",
"md5": "019f4fe21ad941ac6ca6bbf93c404d92",
"sha256": "0531eb8cc02d5b67d8622f47eb4039861cb4697a87ee6db2a98a2e28a7697d51"
},
"downloads": -1,
"filename": "pyfury-0.4.1-cp38-cp38-macosx_12_0_x86_64.whl",
"has_sig": false,
"md5_digest": "019f4fe21ad941ac6ca6bbf93c404d92",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": null,
"size": 970616,
"upload_time": "2023-12-08T17:59:13",
"upload_time_iso_8601": "2023-12-08T17:59:13.931503Z",
"url": "https://files.pythonhosted.org/packages/7a/24/e6d5188d1b1692cd1cff9bf786f7b63353e5dcb6f7690e78bb362327aed4/pyfury-0.4.1-cp38-cp38-macosx_12_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "131728a4b56ba4f2f5937ee7be3561a03f9bac470c830075057dfa8a71d6a562",
"md5": "9f20f6ba20e7c5c07d2cad7b7f0cbb7a",
"sha256": "d928cb07114559e9d0136c3db9438d0902a2d61db7be8738b219c889b87a5b69"
},
"downloads": -1,
"filename": "pyfury-0.4.1-cp38-cp38-manylinux1_x86_64.whl",
"has_sig": false,
"md5_digest": "9f20f6ba20e7c5c07d2cad7b7f0cbb7a",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": null,
"size": 1185801,
"upload_time": "2023-12-08T17:14:42",
"upload_time_iso_8601": "2023-12-08T17:14:42.390394Z",
"url": "https://files.pythonhosted.org/packages/13/17/28a4b56ba4f2f5937ee7be3561a03f9bac470c830075057dfa8a71d6a562/pyfury-0.4.1-cp38-cp38-manylinux1_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "84e669a31e652a3ba0c66b094281996287e4beb484dc6bd1886cf894688d953a",
"md5": "652dcb400cc927c2dbe965bb2ba38fd5",
"sha256": "c2cb3e5bc5b06b2543dcc70c68d6ff8182c5d7b1c12ebda53ce39f780d58e826"
},
"downloads": -1,
"filename": "pyfury-0.4.1-cp39-cp39-macosx_12_0_x86_64.whl",
"has_sig": false,
"md5_digest": "652dcb400cc927c2dbe965bb2ba38fd5",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": null,
"size": 975156,
"upload_time": "2023-12-08T18:00:20",
"upload_time_iso_8601": "2023-12-08T18:00:20.937452Z",
"url": "https://files.pythonhosted.org/packages/84/e6/69a31e652a3ba0c66b094281996287e4beb484dc6bd1886cf894688d953a/pyfury-0.4.1-cp39-cp39-macosx_12_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "94fe156a50427c814bdc041b098b011ade55d4bd5c76b74a41aad0ffa8127852",
"md5": "118a48e38cd0ed19c648de0d3b8f9290",
"sha256": "ce95d97ac1411312152f09daaf1f60b378acb33afcca61dfdab1f5375c8d2e73"
},
"downloads": -1,
"filename": "pyfury-0.4.1-cp39-cp39-manylinux1_x86_64.whl",
"has_sig": false,
"md5_digest": "118a48e38cd0ed19c648de0d3b8f9290",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": null,
"size": 1183944,
"upload_time": "2023-12-08T17:15:06",
"upload_time_iso_8601": "2023-12-08T17:15:06.666604Z",
"url": "https://files.pythonhosted.org/packages/94/fe/156a50427c814bdc041b098b011ade55d4bd5c76b74a41aad0ffa8127852/pyfury-0.4.1-cp39-cp39-manylinux1_x86_64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-08 17:59:05",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "pyfury"
}
JSON