Skip to content

Commit

Permalink
Migrated the Federated Language project to a repository dedicated to …
Browse files Browse the repository at this point in the history
…this project.

To: https://github.com/google-parfait/federated-language/
From: https://github.com/google-parfait/tensorflow-federated/

TensorFlow Federated (TFF) is being split into multiple projects to enable the federated language, runtimes, and libraries to evolve independently from each other and from TensorFlow. While TFF already supports both TensorFlow and JAX, multiple projects will make it easier and explicit how to extend support to other frameworks (e.g., PyTorch). Additionally, this change will increase the velocity of using TFF to develop new ideas both internally and externally.

PiperOrigin-RevId: 702437793
  • Loading branch information
michaelreneer authored and copybara-github committed Dec 3, 2024
1 parent 7c40ba8 commit 0ae85a7
Show file tree
Hide file tree
Showing 471 changed files with 12,662 additions and 50,019 deletions.
7 changes: 2 additions & 5 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,8 @@ build --compilation_mode=opt
# build --define=use_fast_cpp_protos=true
build --define=allow_oversize_protos=true

# pybind_abseil does not include a __init__.py, which breaks dependencies when
# bazel produces an empty __init__.py and hides the symbols from the `.so`
# python extension. We must ask bazel not to generate these and rely on
# explicit __init__.py files.
# build --incompatible_default_to_explicit_init_py
# Do not automatically create `__init__.py` in the runfiles of Python targets.
build --incompatible_default_to_explicit_init_py

# Haswell processor and later optimizations. This covers most processors deployed
# today, includin Colab CPU runtimes.
Expand Down
16 changes: 16 additions & 0 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,22 @@ http_archive(
# },
# )

http_archive(
name = "federated_language",
patches = [
"//third_party/federated_language:proto_library_loads.patch",
"//third_party/federated_language:python_deps.patch",
# Must come after `python_deps.patch`, this patches the output of `python_deps.patch`.
"//third_party/federated_language:structure_visibility.patch",
],
repo_mapping = {
"@protobuf": "@com_google_protobuf",
},
sha256 = "de4bbfd93ee10c3797463d6a96911963d8969fb5ed0b264ae4f3f3088013fed4",
strip_prefix = "federated-language-5405fd4b2965e2f7c6c240b386f0540e4114818e",
url = "https://github.com/google-parfait/federated-language/archive/5405fd4b2965e2f7c6c240b386f0540e4114818e.tar.gz",
)

# The version of TensorFlow should match the version in
# https://github.com/google-parfait/tensorflow-federated/blob/main/requirements.txt.
http_archive(
Expand Down
2 changes: 1 addition & 1 deletion docs/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ There are two principal modes of deployment for TFF computations:
* **Native backends**. We're going to refer to a backend as *native* if it is
capable of interpreting the syntactic structure of TFF computations as
defined in
[`computation.proto`](https://github.com/google-parfait/tensorflow-federated/blob/main/tensorflow_federated/proto/v0/computation.proto).
[`computation.proto`](https://github.com/google-parfait/federated-language/blob/main/tensorflow_federated/proto/computation.proto).
A native backend does not necessarily have to support all language
constructs or intrinsics. Native backends must implement one of the standard
TFF *executor* interfaces, such as
Expand Down
4 changes: 2 additions & 2 deletions docs/design/compilation.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,13 @@ support [Computations](#computation) backed by other external runtimes.
### `Computation`

A
[pb.Computation](https://github.com/google-parfait/tensorflow-federated/blob/main/tensorflow_federated/proto/v0/computation.proto)
[pb.Computation](https://github.com/google-parfait/federated-language/blob/main/tensorflow_federated/proto/computation.proto)
is the Proto or serialized representation of the [AST](#ast).

#### TensorFlow Computation

A
[pb.Computation](https://github.com/google-parfait/tensorflow-federated/blob/main/tensorflow_federated/proto/v0/computation.proto)
[pb.Computation](https://github.com/google-parfait/federated-language/blob/main/tensorflow_federated/proto/computation.proto)
that represents a [Computations](#computation) that will be delegated to the
[TensorFlow](execution.md#tensorflow) runtime.

Expand Down
2 changes: 1 addition & 1 deletion docs/federated_core.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ blocks such as `tff.federated_sum`, `tff.federated_reduce`, or

TFF uses an internal language to represent federated computations, the syntax of
which is defined by the serializable representation in
[computation.proto](https://github.com/google-parfait/tensorflow-federated/blob/main/tensorflow_federated/proto/v0/computation.proto).
[computation.proto](https://github.com/google-parfait/federated-language/blob/main/tensorflow_federated/proto/computation.proto).
Users of FC API generally won't need to interact with this language directly,
though. Rather, we provide a Python API (the `tff` namespace) that wraps arounds
it as a way to define computations.
Expand Down
5 changes: 2 additions & 3 deletions examples/custom_data_backend/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,11 @@ cc_library(
hdrs = ["data_backend_example.h"],
deps = [
"//tensorflow_federated/cc/core/impl/executors:data_backend",
"//tensorflow_federated/cc/core/impl/executors:status_macros",
"//tensorflow_federated/cc/core/impl/executors:tensorflow_utils",
"//tensorflow_federated/proto/v0:array_cc_proto",
"//tensorflow_federated/proto/v0:computation_cc_proto",
"//tensorflow_federated/proto/v0:executor_cc_proto",
"@com_google_absl//absl/status",
"@federated_language//federated_language/proto:array_cc_proto",
"@federated_language//federated_language/proto:computation_cc_proto",
"@org_tensorflow//tensorflow/core:framework",
"@org_tensorflow//tensorflow/core:protos_all_cc",
],
Expand Down
6 changes: 3 additions & 3 deletions examples/custom_data_backend/data_backend_example.cc
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,17 @@ limitations under the License
#include <utility>

#include "google/protobuf/any.pb.h"
#include "federated_language/proto/array.pb.h"
#include "tensorflow/core/framework/tensor.h"
#include "tensorflow/core/framework/tensor.pb.h"
#include "tensorflow_federated/cc/core/impl/executors/tensorflow_utils.h"
#include "tensorflow_federated/proto/v0/array.pb.h"

namespace tensorflow_federated_examples {

namespace {

using ::tensorflow_federated::v0::Data;
using ::tensorflow_federated::v0::Type;
using ::federated_language::Data;
using ::federated_language::Type;
using ::tensorflow_federated::v0::Value;

// Constant URIs and values resolved by `DataBackendExample`.
Expand Down
9 changes: 4 additions & 5 deletions examples/custom_data_backend/data_backend_example.h
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,18 @@ limitations under the License
#define THIRD_PARTY_TENSORFLOW_FEDERATED_EXAMPLES_CUSTOM_DATA_BACKEND_DATA_BACKEND_EXAMPLE_H_

#include "absl/status/status.h"
#include "federated_language/proto/computation.pb.h"
#include "tensorflow_federated/cc/core/impl/executors/data_backend.h"
#include "tensorflow_federated/proto/v0/computation.pb.h"
#include "tensorflow_federated/proto/v0/executor.pb.h"

namespace tensorflow_federated_examples {

// An example implementation of `DataBackend` used to show Python interop.
class DataBackendExample : public tensorflow_federated::DataBackend {
public:
absl::Status ResolveToValue(
const tensorflow_federated::v0::Data& data_reference,
const tensorflow_federated::v0::Type& data_type,
tensorflow_federated::v0::Value& value_out) final;
absl::Status ResolveToValue(const federated_language::Data& data_reference,
const federated_language::Type& data_type,
tensorflow_federated::v0::Value& value_out) final;
};

} // namespace tensorflow_federated_examples
Expand Down
8 changes: 1 addition & 7 deletions tensorflow_federated/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -20,19 +20,13 @@ py_library(
"//tensorflow_federated/python/core/environments/jax",
"//tensorflow_federated/python/core/environments/tensorflow",
"//tensorflow_federated/python/core/framework",
"//tensorflow_federated/python/core/impl/computation:computation_base",
"//tensorflow_federated/python/core/impl/federated_context:federated_computation",
"//tensorflow_federated/python/core/impl/federated_context:intrinsics",
"//tensorflow_federated/python/core/impl/federated_context:value_impl",
"//tensorflow_federated/python/core/impl/types",
"//tensorflow_federated/python/core/impl/types:computation_types",
"//tensorflow_federated/python/core/impl/types:placements",
"//tensorflow_federated/python/core/impl/types:typed_object",
"//tensorflow_federated/python/core/templates",
"//tensorflow_federated/python/core/test",
"//tensorflow_federated/python/learning",
"//tensorflow_federated/python/program",
"//tensorflow_federated/python/simulation",
"@federated_language//federated_language",
],
)

Expand Down
64 changes: 32 additions & 32 deletions tensorflow_federated/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,38 +29,38 @@
from tensorflow_federated.python.core.environments import jax
from tensorflow_federated.python.core.environments import tensorflow
from tensorflow_federated.python.core.impl import types
from tensorflow_federated.python.core.impl.computation.computation_base import Computation
from tensorflow_federated.python.core.impl.federated_context.federated_computation import federated_computation
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_aggregate
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_broadcast
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_eval
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_map
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_max
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_mean
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_min
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_secure_select
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_secure_sum
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_secure_sum_bitwidth
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_select
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_sum
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_value
from tensorflow_federated.python.core.impl.federated_context.intrinsics import federated_zip
from tensorflow_federated.python.core.impl.federated_context.intrinsics import sequence_map
from tensorflow_federated.python.core.impl.federated_context.intrinsics import sequence_reduce
from tensorflow_federated.python.core.impl.federated_context.intrinsics import sequence_sum
from tensorflow_federated.python.core.impl.federated_context.value_impl import to_value
from tensorflow_federated.python.core.impl.federated_context.value_impl import Value
from tensorflow_federated.python.core.impl.types.computation_types import FederatedType
from tensorflow_federated.python.core.impl.types.computation_types import FunctionType
from tensorflow_federated.python.core.impl.types.computation_types import SequenceType
from tensorflow_federated.python.core.impl.types.computation_types import StructType
from tensorflow_federated.python.core.impl.types.computation_types import StructWithPythonType
from tensorflow_federated.python.core.impl.types.computation_types import TensorType
from tensorflow_federated.python.core.impl.types.computation_types import to_type
from tensorflow_federated.python.core.impl.types.computation_types import Type
from tensorflow_federated.python.core.impl.types.placements import CLIENTS
from tensorflow_federated.python.core.impl.types.placements import SERVER
from tensorflow_federated.python.core.impl.types.typed_object import TypedObject
from federated_language import Computation
from federated_language import federated_computation
from federated_language import federated_aggregate
from federated_language import federated_broadcast
from federated_language import federated_eval
from federated_language import federated_map
from federated_language import federated_max
from federated_language import federated_mean
from federated_language import federated_min
from federated_language import federated_secure_select
from federated_language import federated_secure_sum
from federated_language import federated_secure_sum_bitwidth
from federated_language import federated_select
from federated_language import federated_sum
from federated_language import federated_value
from federated_language import federated_zip
from federated_language import sequence_map
from federated_language import sequence_reduce
from federated_language import sequence_sum
from federated_language import to_value
from federated_language import Value
from federated_language import FederatedType
from federated_language import FunctionType
from federated_language import SequenceType
from federated_language import StructType
from federated_language import StructWithPythonType
from federated_language import TensorType
from federated_language import to_type
from federated_language import Type
from federated_language import CLIENTS
from federated_language import SERVER
from federated_language import TypedObject
from tensorflow_federated.version import __version__
# pylint: enable=g-importing-member

Expand Down
Loading

0 comments on commit 0ae85a7

Please sign in to comment.