Apache Wayang
@apachewayang
Followers
90
Following
72
Media
5
Statuses
77
Apache Wayang is the API of AI, cross-platform data processing to train large models. Star / fork us on GitHub: https://t.co/pZY15teRwb
Joined April 2021
Apache Wayang isn’t just limited to FedML and Data Federation; it also enhances our taste and lifestyle. Solving real world data science problems!.
0
1
1
RT @mapredit: @apachewayang 1.0.0 build successful! The backbone for AI, decentralized data processing to access data in silos a dev can't….
0
1
0
RT @mapredit: @apachewayang v. 1.0.0 successfully compiled. The backbone for ASI / AGI infrastructure, devs can now reach data which is typ….
0
1
0
LLM's struggle to show their business benefit. Why? Data access. AI needs the right data, in a suitable format at the right time. Read how @apachewayang helps to solve that:
wayang.apache.org
AI systems and applications are widely used nowadays, from assisting grammar spellings to
0
0
1
Our newest blogpost, written by Wayang developer Juri Petersen, explains how python works with Apache Wayang, and how we enable JSON conform UDF within a wayangPlan. #data #opensource #apachewayang.
wayang.apache.org
In the vast landscape of data processing, efficiency and flexibility are
0
0
0
Read part 1 of the blog series, written by Dr. Mirko Kämpf from ecolytiq, about implementing a cross-organization decentralized data processing scenario. #apachewayang #kafka #data .
wayang.apache.org
Intro
0
0
2
EarthObservation / ESA / Copernicus: AGORA plugin (Apache Wayang):
0
1
1
0
0
1
We updated our website -> <-. More documentation, much more easier to contribute and add PRs.
wayang.apache.org
@{siteConfig.tagline}
0
1
2
Apache Wayang now enables AI/ML training and inference in Spark for federated learning at the enterprise level. The team just merged PR388:
github.com
For clustering task: KMeansOperator For classification task: DecisionTreeClassificationOperator For regression task: LinearRegressionOperator
0
2
3
We merged 380: #380 Added a new plugin to support different jdbc platforms and multiple instances of same DBMS. Wayang now adds support for multiple spark and jdbc context in one wayangPlan().
github.com
The current architecture of wayang looks like this I created a new plugin that can reduce code redundancy to add different jdbc platforms and also support multiple instances of same DBMS by simply...
0
0
1
ML4ALL implementation merged, it's the code to our reseach paper, now merged and fully operable.
github.com
First versio of ML4all module. It contains the abstraction we discuss in the paper and two implementations, one for kmeans and one for SGD. More improvements to come.
0
2
2
Now you can run kmeans natively on sparkML! This improvement speeds ML operations massively up. Additionally, with wayang you can have multiple _different_ spark version in one plan, that makes wayang really powerful for cross-spark ML.
github.com
I implemented a naive kmeans operator on sparkml, and add two test cases. It is still in development. Maybe you can give me some advice on the code and design. Thanks!
0
1
2
Apache Wayang: A Unified Data Analytics Framework - research paper now released with Sigmod Record =>
0
3
9
Wayang AgoraEO Plugin: The Framework for Scalable EO Workflows. "We recently proposed AgoraEO, a decentralized, open, and unified ecosystem, where users can find EO elements, compose cross-platform EO pipelines, and execute them efficiently.
0
2
3
RT @TheASF: #CommunityOverCode NA, our flagship event, is only TWO MONTHS away! Join us in Halifax 🇨🇦 for four days of sessions, with track….
0
4
0
RT @doris_apache: #DynamicPartitioning in Apache Doris.It means to automatically create and delete data partitions based on the user-specif….
0
2
0
RT @DatabloomAI: Apache Wayang enables cross platform data processing, star it via: #dataprocessing #bigdata #anal….
github.com
Apache Wayang(incubating) is the first cross-platform data processing system. - apache/incubator-wayang
0
1
0
Wayang can now support simple and basic SPJ queries for decoupled JDBC sources and CSV data sources.
github.com
This PR makes extends the SQL API to also handle joins. Algebraic Joins are transpiled to Wayang Join Operator Since the JoinOperator outputs a Tuple2, the join is fu...
0
0
1
RT @paulk_asert: Since the latest JDK21ea supports virtual threads as standard (not preview), I updated the Groovy/GPars meets virtual thre….
0
22
0