React: Implement Sideways Data Loading

undefined is probably the safest value to assign to data until the observable provides its first value via onNext. For example in Relay we assign different meanings to null (data does not exist) and undefined (not yet fetched), so our ideal default data value would be undefined. The alternative is to provide a new method, eg getInitialData, but I suspect this is unnecessary/overkill.

This is pretty interesting however from the point of view of static typing I'm not so happy of key/value system, their type is pretty much impossible to express.
Why not having observe return a single observable and set/merge the value resolved to this.data :

class Foo {

  observe() {
    return (
      loadUser(this.props.userID)
        .map(user => { user })
  }

  render() {
    if (this.data.user.id !== this.props.userID) {
      // Ensure that we never show inconsistent userID / user.name combinations.
      return <Spinner />;
    }
    return <div>Hello, {this.data.user.name} [{this.props.userID}]!</div>;
  }

}

And for the case of multiple fetch something like :

class Foo {

  observe() {
    return (
     combineLatest(
      loadUser(this.props.userID),
      loadSomethingElse(this.props.somethingElseId),
      (user, somethingElse) => ({ user, somethingElse})
     )
  }

  render() {
    ..
  }

}

This is perhaps a bit more verbose, but it allows to have nice static type :

interface Comp<T> {
  observe(): Observable<T>;
  data: T;
}

Also instead of re executing observe when props/state change we can have access to 'props' 'state' as an observable :

class Foo {

  observe(propsStream) {
    return (
      propsStream
        .flatMap(({ userID }) => loadUser(userId))
        .map(user => { user })
    );
  }

  render() {
    if (this.data.user.id !== this.props.userID) {
      // Ensure that we never show inconsistent userID / user.name combinations.
      return <Spinner />;
    }
    return <div>Hello, {this.data.user.name} [{this.props.userID}]!</div>;
  }
}

The reason is because we don't want to require the use of combinators and understanding RxJS to be able to subscribe to (multiple) Observables. Combining two Observables in this way is quite confusing. In fact, at least for our data sources, we'll probably implement the subscription API but not even include the combinators on the Observables' prototype. That's not a requirement, but you're free to use combinators if you need to.

However, to subscribe to a simple Flux store, you shouldn't need to.

I think that Flow will probably be able to handle this static type using constraints, but I will check with those guys to make sure. I think that it'll be enough to type the data property and then the observe type can be implied.

class Foo extends React.Component {
  data : { user : User, content : string };
  observe() /* implied as { user : Observable<User>, content : Observable<string> } */ {
  }
}

This change is not about going all in on Observables as a way to describe application state. That can be implemented on top of this, like you've done before. This is explicitly not about application state since this method is idempotent. The framework is free to unsubscribe and resubscribe as needed.

cc @ericvicenti

At least in the case of typescript, there would be no way to constraint the return type of observe based on the type of data, at least until something like https://github.com/Microsoft/TypeScript/issues/1295 is implemented.

I'd love to use this for the next version of React DnD, but obviously this requires waiting for React 0.14.
I wonder if I can “polyfill” this for the time being with a higher-order component that sets this.data on a ref instance.. Might be too crazy though.

Would it be possible to observe Promises? Then one could use a Promises tree to resolve data for the whole component tree before the first render! This would be very useful for server-side React.

What are the benefits of making this a first-class API? It could essentially be accomplished using a "higher-order component."

What are the benefits of making this a first-class API? It could essentially be accomplished using a "higher-order component."

Wrapping in 5 HOCs to get 5 subscriptions is a bit unwieldy and harder to understand for beginners. Understanding componentWillReceiveProps is also non-trivial. This fixes both.

I, for one, welcome our new observable overlords.

I wonder if this can help bring https://github.com/chenglou/react-state-stream closer to React's vanilla API

Wouldn't it just take one HOC? In the example in your Medium post, you iterate over stores and subscribe to each.

stores.forEach(store =>
  store.addChangeListener(this.handleStoresChanged)
);

@aaronshaf Depends on use case, for sure. Sometimes it's different kinds of state sources, not just “several stores”. But I can't say on behalf of React team, let's hear what @sebmarkbage says.

Would love some sort of polyfill to play with this now. I didn't get the idea completely, yet. What is the mechanism involved with dealing with future updates. I'll spend some more time understanding it. I think it should be doable with a simple mixin.

(@vjeux told me I should chime in! so here I am.)

I don't mean to promote my own work, but I think this hook is very similar to the getNexusBindings hook in React Nexus. You declare data deps at the component level via a lifecycle hook (which can depend on props).

The API looks like:

class UserDetails {
  getNexusBindings(props) {
    return {
      // binding to data in the datacenter
      posts: [this.getNexus().remote, `users/${this.props.userId}/posts`],
      // binding to data in the local flux
      mySession: [this.getNexus().local, `session`],
    }
  }
}

The binding is applied/updated during componentDidMount and componentWillReceiveProps. In the latter case, the next bindings are diffed with the previous bindings; removed bindings are unsubscribed, added bindings are subscribed. The underlying fetching/updated mechanism is described in the implementation of Nexus Flux. Basically with the same API you can either subscribe to local data (traditional local stores) or remote data (fetch using GET and receive patches via Websockets/polyfill). You could actually subscribe to data from another window (using postWindow) or a WebWorker/ServiceWorker but I still haven't found a truly useful use case for this.

Long story short, you synchronously describe data deps at the component level using a Flux abstraction and the hooks make sure your dependencies are automatically subscribed, injected on updates, and unsubscribed.

But it also comes with a nice feature: the exact same lifecycle functions are leveraged to perform data prefetching at server-side rendering time. Basically, starting from the root and recusrively from there, React Nexus pre-fetches the bindings, renders the component, and continues with the descendants until all the components are rendered.

@aaronshaf @gaearon The benefit of making it first class is:

1) It doesn't eat away at the props namespace. E.g. the higher-order component doesn't need to claim a name like data from your props object that can't use for anything else. Chaining multiple higher order components keeps eating up more names and now you have to find a way to keep those names unique. What if you're composing something that might already be composed and now you have a name conflict?

Besides, I think that best-practice for higher-order components should be to avoid changing the contract of the wrapped component. I.e. conceptually it should be the same props in as out. Otherwise it is confusing to use and debug when the consumer supplies a completely different set of props than is received.

2) We don't have to use state to store the last value. The data concept is similar to props in the sense that it is purely a memoization. We're free to throw it out at any point if we need to reclaim the memory. For example, in an infinite scroll we might automatically clean up invisible subtrees.

@RickWong Yes, it would be fairly trivial to support Promises since they're a subset of Observables. We should probably do that to be unopinionated. However, I would still probably recommend against using them. I find that they're inferior to Observables for the following reasons:

A) They can't be canceled automatically by the framework. The best we can do is ignore a late resolution. In the meantime, the Promise holds on to potentially expensive resources. It is easy to get into a thrashy situation of subscribe/cancel/subscribe/cancel... of long running timers/network requests and if you use Promises, they won't cancel at the root and therefore you have to just wait for the resources to complete or timeout. This can be detrimental to performance in large desktop pages (like facebook.com) or latency critical apps in memory constrained environments (like react-native).

B) You're locking yourself into only getting a single value. If that data changes over time, you can't invalidate your views and you end up in an inconsistent state. It is not reactive. For a single server-side render that might be fine, however, on the client you should ideally be designing it in a way that you can stream new data to the UI and automatically update to avoid stale data.

Therefore I find that Observable is the superior API to build from since it doesn't lock you in to fix these issues if you need to.

@elierotenberg Thanks for chiming in! It does seem very similar indeed. Same kind of benefits. Do you see any limitations with my proposal? I.e. there something missing, that React Nexus has, which you couldn't build on top of this? Would be nice if we didn't lock ourselves out of important use cases. :)

From the server-rending standpoint, it is important that we're able postpone the final renderToString until the Observable/Promise has been resolved with data that could be fetched asynchronously. Otherwise, we're still in the position of having to do all asynchronous data fetching outside of React without knowing which components will be on the page yet.

I believe react-nexus does allow asynchronous loading to happen within a component before continuing down the render tree.

Yes, react-nexus explicitly separates:
1) binding declaration as getNexusBindings (which is a synchronous, side-effect free lifecycle method, similar to render - actually it used to be name renderDependencies but I thought it was confusing),
2) binding subscription/update as applyNexusBindings (which is synchronous and diffs the previous nexus bindings to determine which new bindings must be subscribed and which ones must be unsubscribed)
3) binding prefetching as prefetchNexusBindings (which is asynchronous and resolves when the "initial" (whathever this means) value is ready)

ReactNexus.prefetchApp(ReactElement) returns a Promise(String html, Object serializableData). This hook mimicks the construction of the React tree (using instantiateReactComponent) and recursively constructs/prefetches/renders the components. When the whole component tree is 'ready', it finally calls React.renderToString, knowing that all the data is ready (modulo errors). Once resolved, the value of this Promise can be injected in the server response. On the client, the regular React.render() lifecycle works as usual.

If anybody wants to play around with this kind of API, I made a really dumb polyfill for observe as a higher order component:

import React, { Component } from 'react';

export default function polyfillObserve(ComposedComponent, observe) {
  const Enhancer = class extends Component {
    constructor(props, context) {
      super(props, context);

      this.subscriptions = {};
      this.state = { data: {} };

      this.resubscribe(props, context);
    }

    componentWillReceiveProps(props, context) {
      this.resubscribe(props, context);
    }

    componentWillUnmount() {
      this.unsubscribe();
    }

    resubscribe(props, context) {
      const newObservables = observe(props, context);
      const newSubscriptions = {};

      for (let key in newObservables) {
        newSubscriptions[key] = newObservables[key].subscribe({
          onNext: (value) => {
            this.state.data[key] = value;
            this.setState({ data: this.state.data });
          },
          onError: () => {},
          onCompleted: () => {}
        });
      }

      this.unsubscribe();
      this.subscriptions = newSubscriptions;
    }

    unsubscribe() {
      for (let key in this.subscriptions) {
        if (this.subscriptions.hasOwnProperty(key)) {
          this.subscriptions[key].dispose();
        }
      }

      this.subscriptions = {};
    }

    render() {
      return <ComposedComponent {...this.props} data={this.state.data} />;
    }
  };

  Enhancer.propTypes = ComposedComponent.propTypes;
  Enhancer.contextTypes = ComposedComponent.contextTypes;

  return Enhancer;
}

Usage:

// can't put this on component but this is good enough for playing
function observe(props, context) {
  return {
    yourStuff: observeYourStuff(props)
  };
}

class YourComponent extends Component {
  render() {
    // Note: this.props.data, not this.data
    return <div>{this.props.data.yourStuff}</div>;
  }
}

export default polyfillObserve(YourComponent, observe);

Is Observable a concrete, agreed upon thing aside from library implementations? What is the contract, is it simple enough to implement without needing to use bacon or Rxjs? As nice as a first class api for sideloading data would be, it seems weird for React to add an api based on an unspeced/very-initial-specing primitive, given React's steady movement towards plain js. Would something like this ties us to a specific user land implementation?

as an aside why not Streams? I have no horse in the race, but I am honestly wondering; there is already work done on web streams, and of course there is node

Another two to consider: https://github.com/cujojs/most and https://github.com/caolan/highland

@jquense There is active work on a proposal for adding Observable to ECMAScript 7(+) so ideally this would become plain JS. https://github.com/jhusain/asyncgenerator (Currently out-of-date.)

We would not take on a dependency on RxJS. The API is trivial to implement yourself without using RxJS. RxJS is the closest to the active ECMAScript proposal.

most.js seems doable too.

Bacon.js' API seems difficult to consume without taking on a dependency on Bacon because of the use of the Bacon.Event types for separating values.

The Stream APIs are too high-level and far removed from this use case.

Is there some kind of “await before render" option? I mean on the client it’s not necessary to wait for all Observables before rendering, but on the server you’d want to wait for them to resolve so that each component's render() is full, not partial.

[parent] await observe(). full render(). -> [foreach child] await observe(). full render().

In all of my explorations I found that this is the most important lifecycle hook missing in server-side React.

Following up this discussion, I've tried to sum up what React Nexus does in the following post:

Ismorphic Apps done right with React Nexus

Heres' the diagram of the core prefetching routine:

We would not take on a dependency on RxJS. The API is trivial to implement yourself without using RxJS. RxJS is the closest to the active ECMAScript proposal.

:+1: this is the big concern for me, thinking about say, promises where implementing your own is extremely fraught unless you know what you're doing. I think otherwise you end up with an implicit requirement on a specific lib in the ecosystem. Tangentially...one of the nice things from the promise world is the A+ test suite, so even across libraries there was at least an assurance of a common functionality of .then, which was helpful for promise interop before they were standardized.

this is the big concern for me, thinking about say, promises where implementing your own is extremely fraught unless you know what you're doing. I think otherwise you end up with an implicit requirement on a specific lib in the ecosystem.

Completely agreed. Thankfully observables have a really simple contract, and don't even have built-in methods like then so in a way they're even simpler than promises.

They might become more complicated (and slower) if the committee insists that calling next schedules a micro-task like Promises.

That would bother some a lot of pattern are based on the fact that onNext is synchronous in RxJS :/

I think a common Flux store pattern might be to keep a Map of Observables on a per-key basis so that they can be reused. Then clean them up when everyone is unsubscribed.

That way you can do things like: MyStore.get(this.props.someID) and always get back the same Observable.

That way you can do things like: MyStore.get(this.props.someID) and always get back the same Observable.

Would using this.props.key (gone I know) make sense? In most cases you already pass such unique identifier with <... key={child.id} .. />.

That way you can do things like: MyStore.get(this.props.someID) and always get back the same Observable.

That's the pattern I use for React Nexus, too. Store#observe returns a memoized, immutable observer; it is cleaned-up (including relevant backend-specific cleanup mechanism, such as sending an actual "unsubscribe" message or whatever) when all subscribers are gone for at least one tick.

@sebmarkbage @gaearon How would observe work on the server in v0.14?
Would it be able to properly wait for all the observers to resolve before rendering to string similar to how react-nexus does (but built in to react)?

IMO it would be great if components waited for the first observed value before being “ready” for rendering on server.

@gaearon: IMO it would be great if components waited for the first observed value before being “ready” for rendering on server.

Yes, :+1: for asynchronous rendering. In the meantime react-async by @andreypopp is an alternative, but it requires fibers to "hack" React. It would be great if React could support asynchronous rendering out-of-the-box.

Async rendering is something we would like to support but is not part of this issue. L

Probably won't make it in 0.14 unfortunately. Many different designs to consider and refactor needed.

Feel free to create and issue describing the architectural changes to the internals needed to make that happen.

I had the same thought as @gaearon re: react-streaming-state. Given all the potential applications other than side loading, might there be a better name than data? For example, observed would more clearly associate it with the method.

Don't mean to derail with bikeshedding but wanted to throw this out there.

can't wait for observables in React. this should make React reactive as I understand it

I'm experimenting with a similar idea while rewriting react-async, see README.

The notable difference is that I introduce explicit observable/process identity to reconcile processes, similar to how React does with key prop and stateful components.

When id of the named process changes React Async stops the old process instance and starts a new one.

The API looks like:

import React from 'react';
import Async from 'react-async';

function defineFetchProcess(url) {
  return {
    id: url,
    start() {
      return fetch(url)
    }
  }
}

function MyComponentProcesses(props) {
  return {
    user: defineFetchProcess(`/api/user?user${props.userID}`)
  }
}

@Async(MyComponentProcesses)
class MyComponent extends React.Component {

  render() {
    let {user} = this.props
    ...
  }
}

The process API now follows ES6 Promises API syntactically and name-wise but semantically it is not expected for process.then(onNext, onError) to be called only once per process live. It is made to accommodate the most popular (?) use case of fetching data via promises. But to be honest, now I think I need to change it to prevent confusion.

I maybe mistaken but I think the only thing which prevents implementing the proposed (in this issue) API in userland is the lack of the lifecycle hook which executes just before render like componentWillUpdate but with new props and state already installed on the instance.

One thing that hasn't yet been discussed is the handling of the onError callback. If an observable produces an error, that information should be available to the component somehow. Because React handles the actual subscribe(callbacks) call, we'd need a standardized method for it to inject into that callback object.

To provide the most flexibility for application developers, I see two approaches. The first is to have the errors placed on a top-level attribute, similar to this.data. This seems incredibly heavy handed, and further eats up the component's namespace.
The second would allow developers to define their own onError callback as a lifecycle-ish function. If I wanted to have custom error handling for my observables, I could add something like

onObserveError(key, error) {
  // do something with the error
  this.state.errors[key] = error;
  this.setState({ errors: this.state.errors });
}

This is similar to what we've done in the upcoming iteration of Parse+React. We created our own error handling to produce the API we wanted. Errors are added to a private { name => error } map, and components have a top-level public method, queryErrors(), that returns a clone of the map if it's not empty, and null otherwise (allowing for a simple if (this.queryErrors()).

Of course, defining new reserved methods is also a tricky business; I won't pretend it isn't. But we need a way to implicitly or explicitly make errors available to the component when rendering.

@andrewimm The idea is to feed that into a generic error propagation system that bubbles error up the hierarchy until it is handled by an error boundary. https://github.com/facebook/react/issues/2928

This should also handle errors in methods throwing and recover gracefully. Like if the render() method throws. This is significantly more work and will take some time to implement properly but the idea was to unify error handling in this way.

I would argue this should be left outside of react proper, and 1 or 2 key integration points should be coordinated with projects like react-async and react-nexus so this can be cleanly done on top of React proper....

I agree, it seems like having a suggested way to do this is better than
baking this into the framework itself

On Tue, Apr 21, 2015, 11:38 PM Rodolfo Hansen [email protected]
wrote:

I would argue this should be left outside of react proper, and 1 or 2 key
integration points should be coordinated with projects like react-async and
react-nexus so this can be cleanly done on top of React proper....

—
Reply to this email directly or view it on GitHub
https://github.com/facebook/react/issues/3398#issuecomment-95048028.

Over the weekend, I built yet another Flux implementation, called Flexy. In this, check out the code for the store. It exposes a .getObservable method that conforms to the observable API, even though it really has no observables or other Reactive framework being used.

So, I would say that the API is easy enough to create with actual Observables.

That said, don't judge the code to harshly, it was done over a weekend for:

• fun
• understanding
• using js-csp
• using the observe API

Side-note, a system like this and Flux actually makes server-side rendering less painful. Using a similar system as React-Nexus, we can initialize stores and pass them to a React app. We can then monitor the stores and dispatcher and keep re-rendering till no more actions are being fired (all required data is already in the stores).

I would argue that this is the smallest integration point to get the new semantics of stateless data subscriptions. What other integration points would you suggest? Not including async rendering which is a much more complex issue and deserves its own thread, and this hook could be used with async rendering regardless.

Everything else is already possible to implement on top of React on a per-component basis. Note that we're not going to do global injections of plugins since that breaks component reuse across environment so frameworks built on top of React should be contextual to individual components.

What hooks are we missing?

Hey,

To be honest, while new hooks would've made implementation easier, we can certainly achieve sideways data loading without them, as we've demonstrated with react-async and react-nexus.

If anything, exposing and supporting maintaining React components instances lifecycle outside of a mounted React hierarchy would help. In react-nexus I use the internal instanciateReactComponent and call componentWillMount, componentWillUnmount, etc, myself, and I consider this approach brittle (what if instanciateReactComponents relies on internal invariants that change on the next version of React?).

As for the stateless approach, it seems to me that async data fetching _is_ stateful, and thus storing the pending/completed/failed status in some components' state is relevant. When we use react-nexus in real-world apps, we have higher-order components which perform data fetching and inject their fetching state to their children components as props. The inner component is thus "stateless" (which is desirable), and the outer component is "stateful" (which is also desirable, eg. to display a loading spinner or a placeholder).

What other integration points would you suggest?

it seems to me that @ANDREYPOPP asked the right question. isn't the only thing we need to implement this in userland the lifecycle hook before render? that seems like the smallest minimal API change needed, the rest is setting and triggering forceUpdate appropriately as you change data based on whatever input stream/emitter/observable. Unless I'm missing something else special about it (entirely possible)?

I'm not getting drawn into the larger discussion here.
But to answer @sebmarkbage I think one of the most important hook I would like is something that is only needed when not using real observables.

• A hook that can provide functions that can deal with the values pushed by the observable _before_ they are set to data. With real observables, this would just be a .map

If the situation was a little more open, I think there should be hooks to replace the Observable-specific behaviour with custom hooks. This way we could use event-emitters, or CSP channels instead.

It seems to me like the last few comments are saying that the smallest extension point is actually "connect" and "disconnect" lifecycle hooks - which make it easy to attach async data to a component and manage those subscriptions?

Observables could then be fairly trivially built upon that (in or outside core) - but exposing those lifecyle points has a broader appeal?

Is that a reasonable summary?

I am also not yet convinced that this needs to be solved in React itself. I am much more inclined to think that React should provide any necessary hooks to build this functionality on top of React.

But for a moment, let's say we're going with observe(). A few thoughts:

We have this.props, this.state, this.context and now this.data, all as potential sources of new data in render(). This seems excessive to me. Is the idea to separate application state from component state? This might clarify and separate some issues around state, but I feel like the cost of introducing a new input may not outweigh the gains. If we want this.state to be focused solely on component state, why not let the fields in this.data add to this.props or this.context instead?

The name this.data is too generic. Props are data, state is data, and any local variable is data. The name adds no meaning and muddles existing meanings. I would much prefer this.observed or any other name that would actually mean something. So +1 to @matthewwithanm's comment:

might there be a better name than data? For example, observed would more clearly associate it with the method.

If we let observe() execute on the server, we need some kind of hook that will clean up any memory leaks this might cause, because unmounting will never happen.

If we call observe() again every time props or state change (and context?) then observe must be optimized for performance and ideally can't be made expensive accidentally. It becomes a part of the hot path. I like this from React Nexus:

the next bindings are diffed with the previous bindings; removed bindings are unsubscribed, added bindings are subscribed.

I've come to believe that state complicates components and I've been trying to use it less in my own components and lift it up instead. This is why, in addition to concerns @fisherwebdev raised (which I agree with now), I'm not convinced that letting observe depend on state is a good idea. State depends on props, observed depends on state _and_ props.. Isn't it too complicated? I'd rather have observe(props).

I'm also coming to realize that observe should not be dependent on state, mostly because it doesn't need to. As @gaearon points out, it's easy enough to hoist the state to one level above, which just feels cleaner after separating concerns. When observe can potentially depend on state, the logic around handling updates within the component grows significantly more complex; when it only depends on props, you can have fork-less handlers in componentDidMount/componentWillReceiveProps. Less code in the critical path translates to a simpler rendering cycle, and it also reduces the number of unintended updates that re-trigger subscriptions.

+1 for observe(props)

The less we deal with state the better IMO.

I'm of the opinion, that as a library, React should try to be as flexible as possible. I agree that it's not a good idea for observe to depend on state. Yes it can be complicated. Yes, the best practice should be to not depend on state.
But that is a choice that users of React should be allowed to make.
I would recommend, leaving the current API unchanged (other than any possible hooks to add more flexibility, to make it work with more than just observables for example), and proving documentation that explains that using state in the observe method is not recommended.

I believe that everyone wants to do the right thing, and that we want to be pointed in the right direction. Having observe not accepting state makes that easier for the user than to accidentally find something like "this is an antipattern" in the docs.

EDIT: sorry, just realized I was looking for flatMap. I confused myself because I was thinking it would flatten the array of messages, but it is operating at a higher level (the messages observable).

Would the proposed API handle the case where the result of one data field is dependent on the result of another? I.e. you map the first result and return an observable:

observe(props, context) {
  if (!props.params.threadID) {
    return {};
  }

  const observeThread = ThreadStore.observeGetByID(
    {id: props.params.threadID}
  );
  return {
    thread: observeThread,
    messages: observeThread.map(thread => {
      return MessageStore.observeGetByIDs({ids: thread.messageIDs});
    })
  };
}

I'm pretty new to observables in general, so I may be going about this completely wrong. In promise-land this is extremely simple, since returning a promise from then will cause subsequent thens to be based off that promise.

I don't understand the comments that it shouldn't depend on this.state. Encapsulated state surely makes React a lot more complicated but that is also all it is. If there wasn't encapsulated state, we would only need a memoized immediate mode library. If you go all in on "Stores", then yes, you don't need state but that is not what React prescribes that you do.

We have several patterns that requires you to create extra wrappers but for normal usage you shouldn't need to. Other than stores, your observed data always depends on state even if indirectly. I don't think it is bad practice at all to depend on it in observe. E.g.

observe() {
  return { items: Items.getPagedItems({ pageIndex: this.state.currentPage }) };
}

Even if observe didn't depend on state it would still depend on props and context. Even if it didn't depend on context you would still have an indirection that uses context to render a component's props that uses it in observe.

observe would need to get reevaluated every time a render pass comes down because the props could've changed. However, we would definitely diff the resulting observables and not unsubscribe/resubscribe if the same observable is returned. However, we can't do diffing on individual properties (other than with shouldComponentUpdate) so ideal you should implement your own cache using a Map as a power feature. That way you can return the same observable to multiple components in the tree. E.g. multiple components loading the same user. Even if you don't, you just recreate the Observable and eventually hit the bottom cache. This is just how React reconciliation works anyway. It's not that slow.

This observe hook isn't designed to make React completely reactive in the sense that state is captured in Observables. It is currently a key design goal is to avoid trapping state in closures and combinators and instead have a nice clean separate state-tree that can be frozen and revived, and potentially shared across workers.

Which brings me to my final point...

We certainly don't _need_ to add this to the core library. The original "public" interface was mountComponent/receiveComponent and you could build your whole composite component system on top of it. However, not many people used that it is much more powerful to raise the abstraction bar since we can now build other things that are enabled by a higher abstraction bar. Such as component-wide optimizations.

React's primary purpose is to create a contract between components so that different abstractions in the ecosystem can co-exist. An important part of that role is to raise the level of abstraction for common concepts so that we can enable new cross-component features. E.g. saving all the state of a subtree, then revive the subtree. Or possibly including automatic unmounting on the server or changing the timing aspects of reconciliation on the server.

It is also important to provide some batteries included to make all this palatable and uniform.

It is important to realize that micro-modularization (like adding a new life-cycle hook) isn't strictly a pure win over building it into the framework. It also means that it is no longer possible to reason about system wide abstractions.

I wish something like this was in the docs as “philosophy / design goals / non-goals”.

React's primary purpose is to create a contract between components so that different abstractions in the ecosystem can co-exist. An important part of that role is to raise the level of abstraction for common concepts so that we can enable new cross-component features.

Love this. I agree with @gaearon that it'd be nice if this were in some sort of doc.

We certainly don't need to add this to the core library.... However, not many people used that it is much more powerful to raise the abstraction bar since we can now build other things that are enabled by a higher abstraction bar. Such as component-wide optimizations.

I feel reticence (for me at least) is not in adding another API, but adding one that depends on a non-language (still being defined) construct to work. This could totally work out fine, but I worry about the issues Promise libraries face where no promise library (even spec complaint ones) can trust each other, which leads to unnecessary wrapping and defensive work to make sure they resolve correctly, which limits optimization opportunities. Or worse yet you get stuck like jQuery with a broken implementation that can never change.

@jquense I completely agree. I wanted to add this hook a long time ago. (Original experiment: https://github.com/reactjs/react-page/commit/082a049d2a13b14199a13394dfb1cb8362c0768a )

The hesitation two years ago was that it was still too far away from standardization. I wanted a standard protocol before we added it too the core.

I think we're getting to a point where a lot of frameworks agree on the need for something like Observables and standardization is reaching a point where a palatable API has been proposed. I'm sure we'll need to end up tweaking it a bit but as long as the high level architecture works it should be swappable and eventually converge.

I think what happened with Promises is that the API and debugging story was severely lacking in certain areas that Observables doesn't suffer from. It is a more complete story out-of-the-box where as Promises had to standardize a minimal incomplete solution.

The only difference in opinion re: Observables I've observed (couldn't resist, sorry) is Zalgo potential. Whether or not Observables may push value synchronously in response to a subscription. Some folks seem against it, but React's use of Observables will depend on this as far as I understand. Can you comment on that?

Generally I don't find Zalgo to be a problem with Observables because consumer is always in control and may opt for always-async with something like observeOn.

Glad that there's finally some consensus on this. I personally prefer channels over Observables, but if Observables are going to be added to the language, I agree there is no need to wait any longer.

That said, let's make sure that we keep the API open enough to work with non-Observables that conform to the basic API.

I don't find Zalgo to be a problem either. However with Observables you can use a scheduler to ensure async if desired, the default scheduler is now async so you can use that as needed.

@sebmarkbage I think you've addressed most of my concerns and I see now the benefit of adding this to the framework. However, can you comment on this.data -- (1) can/should we fold those fields into props/context/state or (2) can we rename it?

Slightly bikeshed-y but going for it anyway...the Zalgo issue isn't really one about whether it matters in terms of API expectation, it is one of observable interop and ease of implementation. Not having early agreement about Zalgo that has put the Promise library world in the annoying position of having to be super defensive when dealing with Promises from other libs. (my above point repeated below)

...where no promise library (even spec complaint ones) can trust each other, which leads to unnecessary wrapping and defensive work to make sure they resolve correctly

Because early promises didn't all comply with async resolution we are in a position now where even tho it is speced libraries can't assume thenables are trust worthy, killing potential optimizations. This strikes me as particularly relevant here, where React will not be providing an Observable implementation to use (who would want that anyway?), and we are most likely years away from being able to rely solely on a browser provided Observable, so easy library interop is important. Plus to @gaearon's point, if React depends on sync calls and it is spec'ed to always be async that puts us in a jquery like position of being stuck with a rogue implementation.

I completely agree. I wanted to add this hook a long time ago. The hesitation two years ago was that it was still too far away from standardization. I wanted a standard protocol before we added it too the core.

I am glad its being attended too and thought about, that is certainly comforting. :) and in general I think the early adoption of promises has been worth the cons i'm discussing here, so don't take my concern as dislike, or disapprove, I am pretty excited at the prospect of a first class API for this and I also see how Observable is really a good/most reasonable choice here for it.

"We should use the RxJS contract of Observable since that is more in common use and allows synchronous execution, but once @jhusain's proposal is in more common use, we'll switch to that contract instead."

Just to add a little bit more context. There is a Reactive Streams initiative (http://www.reactive-streams.org/) to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols.

The current leading implementations are f.e. Akka Streams or RxJava. I don't know if RxJs already comply to the same interface now, the current interface for Subscriber is onSubscribe(Subscription s), onNext(T t), onCompleted(), onError(Throwable t).

Can you shed more light on what is @jhusain's proposal?

I don't know if React should strictly comply to this initiative because if I need I can probably put RxJs (assuming it will comply) in between and adapt to React interface and let more advanced concepts like back-pressure to RxJs (though I would prefer not have to adapt much).

Is there any position or goal regarding this initiative?

@vladap I believe this is the mentioned proposal from @jhusain

I have read through @jhusain and I'm not sure about the motivation to possibly move to this spec in the future. Is there any specific advantage?

Reactive-streams spec has larger support and is already on version 1.0. Because RxJava already implements this spec I assume RxJs will follow (but haven't checked).

This blog sums up the interfaces with some examples using Akka streams.

I can see some advantage to have the same interfaces on the backend and the frontend, mainly because I work on both. Possibly it might help to cooperate between backend and frontend groups but on the other hand I assume that websocket or sse are the actual integration points for streaming.

I can't find the implementators list on www.reactive-streams.org right now but the last time I checked it was:

Björn Antonsson – Typesafe Inc.
Gavin Bierman – Oracle Inc.
Jon Brisbin – Pivotal Software Inc.
George Campbell – Netflix, Inc
Ben Christensen – Netflix, Inc
Mathias Doenitz – spray.io
Marius Eriksen – Twitter Inc.
Tim Fox – Red Hat Inc.
Viktor Klang – Typesafe Inc.
Dr. Roland Kuhn – Typesafe Inc.
Doug Lea – SUNY Oswego
Stephane Maldini – Pivotal Software Inc.
Norman Maurer – Red Hat Inc.
Erik Meijer – Applied Duality Inc.
Todd Montgomery – Kaazing Corp.
Patrik Nordwall – Typesafe Inc.
Johannes Rudolph – spray.io
Endre Varga – Typesafe Inc.

Maybe I go too far here but I believe that the larger context can help in future decisions.

@vladap From what I understand and what I see on the github issues @jhusain is already working with them, so I guess we will not have so much problem.
From an interface perspective, from what I also could grasp in different github issues and other spec document, observer will certainly respect the generator interface so something like :

{
  next(value),
  throw(e),
  return(v)
}

Simply implementing a very basic observable with a single 'subscribe' method that respect that interface should be secure for react.

It looks like just a different naming with the same functionality, in that sense it is fine. I would probably prefer the same naming as in the spec but in the end I don't care that much as far as these methods do the same.

I can''t evaluate missing equivalent to onSubscribe(). In the blog I have mentioned author says that it is a key to control back-pressure. I don't know it has other use cases. From this I assume that React doesn't care about controlling back-pressure or there is another strategy for it. It is a complex thing hence I understand it is not React concern.

Do I understand correctly that the strategy is something along the lines - either the app is complex and there can arise back-pressure problems then use something in between to solve it, like RxJS, or you connect React component directly to f.e. websocket and you don't have back-pressure problems because the app is simple and have slow updates.

Where can I find proposed observable interfaces for the future ECMAScript? If there any already.

The current proposal can be found here:

https://github.com/jhusain/asyncgenerator

JH

On May 7, 2015, at 2:32 AM, vladap [email protected] wrote:

Where can I find proposed observable interfaces for the future ECMAScript? If there any already.

—
Reply to this email directly or view it on GitHub.

The Reactive Streams Proposal (RSP) goes further than the TC-39 proposal because it introduces an Observable that handles back pressure. The RSP Observable is optimized for efficiently sending streams across the network, while respecting backpressure. It's based in part on the work done in RxJava, which is a very impressive piece of engineering (full disclosure: it was designed by colleague at Netflix, Ben Christensen).

The primary reason for the decision to standardize the more primitive Observable type is caution. The more primitive Observable is the dual of the ES2015 Iterable contract, which provides us with valuable guarantees that the type is at least as flexible as a type already standardized in ES2015. Furthermore there are a wide variety of use-cases in JS for Observables which do not require backpressure. In the browser the DOM is the most common sink for push streams, and acts effectively like a buffer. Given that the RSP type is more complex, our approach is to standardize the more primitive type first, and then leave room to implement the more advanced type later. Ideally we'd wait until it was validated in user-land.

FYI RxJS currently has no plans to implement RSP Observable.

JH

On May 7, 2015, at 2:30 AM, vladap [email protected] wrote:

It looks like just a different naming with the same functionality, in that sense it is fine. I would probably prefer the same naming as in the spec but in the end I don't care that much as far as these methods do the same.

I can''t evaluate missing equivalent to onSubscribe(). In the blog I have mentioned author says that it is a key to control back-pressure. I don't know it has other use cases. From this I assume that React doesn't care about controlling back-pressure or there is another strategy for it. It is a complex thing hence I understand it is not React concern.

Do I understand correctly that the strategy is something along the lines - either the app is complex and there can arise back-pressure problems then use something in between to solve it, like RxJS, or you connect React component directly to f.e. websocket and you don't have back-pressure problems because the app is simple and have slow updates.

—
Reply to this email directly or view it on GitHub.

Many thanks for the valuable details. It makes a lot of sense.

@gaearon I sorta copied you. I wanted to use ParseReact with ES6 classes so I needed to reimplement the mixin's observe api as a higher-order component.

https://gist.github.com/amccloud/d60aa92797b932f72649 (usage below)

• I allow observe to be defined on the component or passed into the decorator. (I might merge the two)
• I merge in observables as props (no this.data, or this.props.data)

@aaronshaf @gaearon The benefit of making it first class is:

1) It doesn't eat away at the props namespace. E.g. the higher-order component doesn't need to claim a name like data from your props object that can't use for anything else. Chaining multiple higher order components keeps eating up more names and now you have to find a way to keep those names unique. What if you're composing something that might already be composed and now you have a name conflict?

Besides, I think that best-practice for higher-order components should be to avoid changing the contract of the wrapped component. I.e. conceptually it should be the same props in as out. Otherwise it is confusing to use and debug when the consumer supplies a completely different set of props than is received.

Instead of a HOC, it can be a regular component:

import Observe from 'react/addons/Observe';

class Foo {
  render() {
    return (
      <Observe
        render={this.renderData}
        resources={{
          myContent: xhr(this.props.url)
        }} />
    );
  }

  renderData({ myContent }) {
    if (myContent === null) return <div>Loading...</div>;
    return <div>{myContent}</div>;
  }
}

Because you control the function passed as render prop, there is no way for the names to collide. On the other hand, it does not pollute the owner's state.

What am I missing here?

This could even be less verbose if Observe component just grabbed Observables from its props:

render() {
  return (
    <Observe myContent={Observable.fetch(this.props.url)}
             render={this.renderData} />
  );
}

renderData({ myContent }) {
  if (myContent === null) return <div>Loading...</div>;
  return <div>{myContent}</div>;
}

What's also nice about this is it'll avoid re-subscriptions if shouldComponentUpdate returned false because we won't get into render at all.

Finally, one could write a decorator for render that wraps it into an Observe component:

@observe(function (props, state, context) {
  myContent: Observable.fetch(props.url)
})
render({ myContent }) {
  if (myContent === null) return <div>Loading...</div>;
  return <div>{myContent}</div>;
}

I would prefer to keep render as a pure rendering function instead of injecting data-fetching logic in it.
The initial proposal seems good in my opinion. It's very close of how state works with rx-react and it will allows to separatie state management from data-fetching logic which seems very coherent.

The only thing that bother me is the use of map of observable instead of one observable, because it does not give the user the possibility to choose how observable are composed but this is a minor issue.

It's not really injecting the data fetching logic, just saves some typing. It will desugar to the version above, which just renders <Observe /> component. It's not unusual to render stateful components, so I don't think it makes render any less pure than it is now.

I tried to combine the best from #3858 and this proposal.

In any HOC approaches, the benefit is explicitness, but the downsides are described by @sebmarkbage: the props names may conflict at some point.

In the current proposal, the benefit is explicitness, but the negative side is the more complicated lifecycle and larger core component API surface.

In #3858, the benefit is colocating "memoized render" dependencies with rendering itself (they're not used anywhere else so it makes sense), but I'm concerned about the “looks sync but is async”, and not understanding how it can work with immutable models if it relies on this so much. It also rubs me wrong in the React-was-easy-to-reason-about way because there's nothing easy about reasoning about manually tracking changes and coupling the data sources to React (or wrapping them to work with React). I understand there exists a pressure to implement something both performant and reducing the boilerplate though.

In my proposal, I'm keeping the colocation and explicitness, but:

• <Observe /> (or the observe() decorator that wraps render with <Observe />) is just an add-on, I'm not proposing _any_ changes to React core.
• Everyone can implement their own observing logic for their use cases. You can have your own <Observe /> that uses just one observable, if that's what you prefer. You may or may not use the decorator.
• The component lifecycle stays the same.
• There are no prop conflicts because the data is passed as a parameter.
• We're dogfooding by solving the problems with the tools we have.
• For reducing the boilerplate, we use boilerplate reduction tools (decorators) instead of introducing new core concepts.

Great discussion and work all around here, much respect. :)

I agree that this doesn't merit making it into core. Perhaps an add-on gives this proposal enough traction that folks can try to converge and standardize on it before committing more fully. That being said, I find this proposal better than https://github.com/facebook/react/issues/3858 and https://github.com/facebook/react/pull/3920 for its minimalism.

This is something I've been using on side projects (so grains of salt) - it's similar to @elierotenberg's awesome work but doesn't takeover the lifecycle, since this app isn't 100% in React and has to interop.

Brace yourself for CoffeeScript and mixins, or keep squinting until this looks like ES6 if you prefer. :)

_ = require 'lodash'

module.exports = DeclareNeedsMixin = 
  componentDidMount: ->
    @needsConsumerId = _.uniqueId @constructor.displayName
    @sinkNeeds @props, @state

  componentWillUpdate: (nextProps, nextState) ->
    @sinkNeeds nextProps, nextState

  componentWillUnmount: ->
    @props.flux.declareNeeds @needsConsumerId, []

  sinkNeeds: (props, state) ->
    if not @declareNeeds?
      return console.warn 'Missing method required for DeclareNeedsMixin: `declareNeeds`', @

    needs = @declareNeeds props, state
    props.flux.declareNeeds @needsConsumerId, needs

  # Intended to be overridden by the host class.
  # Returns a set of facts, stored as an array.
  # Yes, immutable data is awesome, that's not the point here though. :)
  # Facts are serializable data, just values.
  # declareNeeds: (props, state) ->
  #   []

And used like this:

module.exports = EmailThreads = React.createClass
  displayName: 'EmailThreads'
  mixins: [DeclareNeedsMixin]

  propTypes:
    flux: PropTypes.flux.isRequired

  declareNeeds: (props, state) ->
    [Needs.GmailData.myThreads({ messages: 20 })]

  ...

So declareNeeds is a one-way function of props and state to a description of what this component needs. In the actual implementation here, the receiving end of @props.flux.declareNeeds, which is set up at a top-level component, sinks these needs into a ProcessSink object. It batches as it likes, de-dupes needs across components that share the same flux, and then performs side-effects to meet those needs (like connecting to a socket or making HTTP requests). It uses reference counting to clean up stateful things like socket connections when there are no components who need them anymore.

Data flows through from stateful bits like socket events and requests into the dispatcher (and then to Stores or wherever) and back to components to meet their needs. This isn't synchronous and so all components handle rendering when data is not yet available.

I'm sharing this here only as another voice that exploring these kinds of solutions are possible in user-space, and that the current API is serving that kind of experimentation really well. In terms of a minimal step core could take to support interop between different approaches, I think @elierotenberg nailed it:

If anything, exposing and supporting maintaining React components instances lifecycle outside of a mounted React hierarchy would help.

As for the stateless approach, it seems to me that async data fetching is stateful, and thus storing the pending/completed/failed status in some components' state is relevant.

@elierotenberg and @andrewimm make an excellent point about first-class error handling. @sebmarkbage I think your instinct towards the minimal interop point is right on, but I'm not sure how adding semantics for errors to bubble up the component tree fits that requirement. There needs to be an equally simple story here for how to access values from onError and onCompleted, even if it's just that slots in this.observed holds the last value of either next/error/completed like { next: "foo" }. Without supporting the observable contract as a first-class feature, I'm a bit skeptical about this proposal making the cut.

And since this is the internet, and one of my first times chiming in here: the React issues feed is some of the best reading and an all-around source for great work and ideas. :+1:

smells like bindings to me.

I'm not sure how this relates to the current proposal/implementation, but I've found that enabling composition and higher-order manipulation is actually a key feature of data dependency tracking, especially if you use a reactive data source (flux, flux over the wire, or anything else than provides you with updates).

Take the following example with react-nexus@^3.4.0:

// the result from this query...
@component({
  users: ['remote://users', {}]
})
// is injected here...
@component(({ users }) =>
  users.mapEntries(([userId, user]) =>
    [`user:${userId}`, [`remote://users/${userId}/profile`, {}]]
  ).toObject()
))
class Users extends React.Component {
  // ... this component will receive all the users,
  // and their updates.
}

All in all, I wonder if there should be a data binding in the component API at all. It seems to me that higher-order-components-returning decorators provide a very nice way of expressing data binding without polluting the component methods namespace.

However, as @sebmarkbage noted, there is a risk of pollution the props namespace instead. For now, I'm using a props transformer decorator (react-transform-props) to clean up/rename props before passing them to the inner component, but I acknowledge it might become problematic in the future if higher-order-components become more commonplace and the risks of name clashing increases.
Could this be solved by using symbols are property keys? Does propTypes checking support Symbol-keyed props? Will JSX support computed inline props keys (although one can always use computed properties + object spread)?

Sorry if this gets a little offtopic but it seems to me that we still have to find the right sweet abstraction/API for expressing data deps at the component level.

my 2¢

I've been having much fun with the 'children as a function' pattern for values that change over time. I believe @elierotenberg first came up with it? My current usage is to model springs (via rebound) on react-springs. Example -

<Springs to={{x: 20, y: 30}} tension={30}>
  {val => <div style={{left: val.x, top: val.y}}>moving pictures</div>}
</Springs>

No props collision, and no usage of the owner's state. I can also nest multiple springs, and react manages all the hard bits. These 'observables' (ha!) could also accept onError, onComplete and other props (graphql queries?).

A rough attempt to sketch this out for 'flux'

<Store 
  initial={0}
  reduce={(state, action) => action.type === 'click'? state+1 : state} 
  action={{/* assume this comes as a prop from a 'Dispatcher' up somewhere */}}> 
    {state => <div onClick={() => dispatch({type: 'click'})}> clicked {state} times</div>}
</Store>

We used this pattern, what we called _render callbacks_, at Asana but are ultimately moving away from it.

Pros

• No potential for collision of props.
• Easy to implement both as the author of StoreComponent and the user of StoreComponent

Cons

• shouldComponentUpdate is extremely difficult to implement since the render callback may close over state. Imagine we had a component tree of A -> Store -> B. A has a counter in its state which is accessed during the render callback as props for B. If A updates because of the counter and the Store does not update, B will have a stale version of the counter. As a result, we were forced to always update the Store.
• Testing components in isolation became very difficult. Inevitably, developers put complex logic in the render callbacks for which component to render and wanted to test the logic. The natural way to do this was to render the whole tree and test _through_ the store component. This made it impossible to use the shallow renderer.

We're now moving to a model where the StoreComponent takes a ReactElement and when the store receives new data, the store clones the ReactElement overriding a specific prop. There are many ways to do this pattern such as taking a Component constructor and props. We went with this approach because it was the easiest to model in TypeScript.

Terrific points, can't think of a way around either without breaking the 'sideways' requirement.

@threepointone Sounds exactly like one of the implementation proposals to power https://github.com/reactjs/react-future/pull/28

@pspeter3 in your example, is there a serious drawback to making Store always update / shouldComponentUpdate: ()=> true? I'm thinking its 'children' would have been rendered without Store in the chain anyway. Thanks for your time!

@threepointone That is exactly what we did for our boundaries. It was unclear what the impact was exactly but there were perf worries from members of the team. The worries combined with the difficulty testing compelled the switch to using React.cloneElement(this.props.child, {data: this.state.data})

@pspeter3 the testing angle is definitely a problem. What if shallowRenderer recognized 'render callbacks'? Would that help?

Ps- 'render callback' :thumbs_up:

@sebmarkbage the current discussion over on es-observable is that subscribe would be guarenteed async, with a [Symbol.observer] method provided to shortcut and subscribe synchronously, though the validity of having both sync/async apis is currently being debated.

I had chimed in on this ticket with the use case mentioned above in favor of synchronous subscription, didn't know if you might have anything to add there.

I think the ideas here are a very clean pattern for handling external state, though after playing around with it for a bit, I'm leaning in favor of the HOC approach.

Also @gaearon I simplified your HOC bit above, using a static - ComposedComponent.observe and using this.state rather than this.state.data - https://gist.github.com/tgriesser/d5d80ade6f895c28e659

It looks really nice with the proposed es7 decorators:

@observing
class Foo extends Component {
  static observe(props, context) {
    return {
      myContent: xhr(props.url)
    };
  }
  render() {
    var myContent = this.props.data.myContent;
    return <div>{myContent}</div>;
  }
}

The class decorator could even go so far as adding a getter for data to make it come very close to the original proposed API (minus local state affecting observable subscriptions, which I agree is a good thing - much less noise around subscribe / unsubscribe).

lack of synchronous subscription could be a huge problem.

I think I've got it how to solve "the state problem" and "Sideways Data Loading" (derivative data) in a consistent way. It makes it in a stateless "React-way". I've found a way how to maintain state consistency at any point of time and it fits the pattern UI = React(state). Unlikely I'm out of hands to make it completely bullet-proof, add more examples and make good presentation. https://github.com/AlexeyFrolov/slt . On other hand it's well tested and I'm using it in my production projects iteratively. Clever minds are welcome to contribute.

Hi all,

Its funny that I didn't stumble in this thread before, as at our company we were running into the same problems addressed by this proposal half a year ago.
We started using React for a large scale project (think of an editor with the complexity of Microsoft Visio, with very cyclic data). Vanilla react countn't keep up with our performance demands,
and flux was a bit of a no-go for us due its large amount of boilerplate and error proneness of all the subscriptions. So we figured out that we needed observable data structures as well.

As we couldn't find anything available that was ready to use, we built our own observables lib, based on the principles of the knockout observables (especially: automatic subscriptions).
This fits actually very neatly in the current lifecycle of React components, and we didn't need weird hacks or even child rendering callbacks (the ObserverMixin used below is about 10 loc).
This improved our DX a lot and worked so tremendously well for our team, that we decided to publish it as open source. In the mean time it is quite battle proven (it provides an ES7 observable array polyfill for example) and highly optimized.
Here is a short timer example, (also available as JSFiddle), IMHO the DX couldn't be much better... :relieved:

var store = {};
// add observable properties to the store
mobservable.props(store, {
    timer: 0
});

// of course, this could be put flux-style in dispatchable actions, but this is just to demo Model -> View
function resetTimer() {
    store.timer = 0;
}

setInterval(function() {
    store.timer += 1;
}, 1000);

var TimerView = React.createClass({
    // This component is actually an observer of all store properties that are accessed during the last rendering
    // so there is no need to declare any data use, nor is there (seemingly) any state in this component
    // the combination of mobservable.props and ObserverMixin does all the magic for us.
    // UI updates are nowhere forced, but all views (un)subscribe to their data automatically
    mixins: [mobservable.ObserverMixin],

    render: function() {
        return (<span>Seconds passed: {this.props.store.timer}</span>);
    }
});

var TimerApp = React.createClass({
    render: function() {
        var now = new Date(); // just to demonstrate that TimerView updates independently of TimerApp
        return (<div>
            <div>Started rendering at: {now.toString()}</div>
            <TimerView {...this.props} />
            <br/><button onClick={resetTimer}>Reset timer</button>
        </div>);
    }
});

// pass in the store to the component tree (you could also access it directly through global vars, whatever suits your style)
React.render(<TimerApp store={store} />, document.body);

For more details about this approach, see this blog. BTW, I'll make sure a decorator and/or container will be added to the lib, for those using ES6 classes.

Saddly, I didn't see this thread before react-europe, otherwise we would have had a nice opportunity to discuss React & observables. But big thanks for the inspiring talks! :+1: I especially loved the abstractions of GraphQL and the thought work behind Redux :)

@mweststrate I feel community will end up with need to choose between the "Observables" and the "Immutable data" solutions ultimately. Maybe we need to mix in some way to have the strengths of both approach in the one solution (https://github.com/AlexeyFrolov/slt/issues/4). In my solution I've implemented the "Immutable data" approach with focus on the consistency of the state at any point of time. I'm planning to support Observables and Generators as well. This is an example of rules with is using to fetch "derivative" or "supplement" data (such a page body, assets, recommendations, comments, likes) and maintain the consistency of the app state.

https://github.com/AlexeyFrolov/slt#rules-example

It's a complex real-world (my production code) example that shows how to handle API redirects with a Location header.

import r from "superagent-bluebird-promise";
import router from "./router";

export default {
    "request": function (req)  {
        let route = router.match(req.url);
        let session = req.session;
        route.url = req.url;
        return this
            .set("route", route)
            .set("session", req.session);
    },
    "route": {
        deps: ["request"],
        set: function (route, request) {
            let {name, params: { id }} = route;
            if (name === "login") {
                return this;
            }
            let url = router.url({name, params: {id}});
            let method = request.method ? request.method.toLowerCase() : "get";
            let req = r[method]("http://example.com/api/" + url);
            if (~["post", "put"].indexOf(method)) {
                req.send(request.body);
            }
            return req.then((resp) => {
                let ctx = this.ctx;
                let path = url.substr(1).replace("/", ".");
                if (!resp.body) {
                    let location = resp.headers.location;
                    if (location) {
                        ctx.set("request", {
                            method: "GET",
                            url: location.replace('/api', '')
                        });
                    }
                } else {
                    ctx.set(path, resp.body);
                }
                return ctx.commit();
            });
        }
    }
}

In the rest it the same approach as yours, except the is no direct binding to the React (it's not required in my case). I believe we need to join forces somehow to archive the common goal.

I think it's a bad idea because there will be many edge-cases that hard to consider.

From comments above i can list possible scopes that needs to be thinked of by end user each time he wants to create "dataful" component:

• Server-side rendering
• Initialization of .state and .data
• .context, .props, .state and .observe entanglement
• Asynchronous rendering

I think this proposal will lead to error prone, unstable, and hard to debug components.

I'm agree with proposed by @glenjamin connect and disconnect lifecycle hooks.

Apologies if this is off-topic (I can't quite tell). But here's an example of why I think exposing the API for interacting with the component tree would be an interesting way to approach this problem: https://github.com/kevinrobinson/redux/blob/feature/loggit-todomvc/examples/loggit-todomvc/loggit/renderers/precompute_react_renderer.js#L72

This method is my hackery to walk the tree, and so my question is how would having a public API for doing this kind of thing enable innovation here in "sideways data-loading," which I think boils down to "a renderer that doesn't work top-down": https://github.com/kevinrobinson/redux/blob/feature/loggit-todomvc/examples/loggit-todomvc/loggit/react_interpreter.js#L8

But here's an example of why I think exposing the API for interacting with the component tree would be an interesting way to approach this problem.

I believe @swannodette talked about this at ReactConf. I wonder if Om Next does it?

Yep, he suggested the same kind of thing at the first ReactConf. I haven't seen the latest Om.next or heard the EuroClojure talk, but previously Om would use its own structure that users had to build off of in order to do this.

This method is my hackery to walk the tree, and so my question is how would having a public API for doing this kind of thing enable innovation here in "sideways data-loading," which I think boils down to "a renderer that doesn't work top-down"

I think this is roughly equivalent to the shallow rendering found in test utils - I mentioned making this a first-class API as a peer of DOM & String rendering to @sebmarkbage at ReactEurope - the 0.14 changes seem to pave the way nicely for this.

Initial reaction is that it might be a bit low level - but in a similar way to the extensible web stuff I think this would make it easier to experiment in user space.

I agree, if we have a way to render to get the tree then we can walk it and find all the data needs and pass them down.

Getting access to the entire virtual DOM tree is an amazingly powerful and useful feature that I would love to get access to, even if that is treated as a completely separate issue.
I think it makes a ton of sense considering the path React is taking with 0.14 onwards.

Given the complexity of observables, I humbly suggest the fine gentlemen in this thread to look at Meteor's implementation of reactive data: https://github.com/meteor/meteor/wiki/Tracker-Manual. Reconciling it with React takes literally 3-5 lines of code in the componentWillMount method, something like:

  componentWillMount() {
    if (typeof this.getState === 'function') {
      Tracker.autorun(() => {
        // Assuming this.getState() calls some functions that return
        // reactive data sources
        this.setState(this.getState());
      });
    }
  }

I don't know if Tracker (which is easily extracted as a separate library) obviates the need for observable support in React, but it sure seems that way.

MOBservable follows a very similar pattern to refresh a component side-ways once some observable values are changed, so it seems that the current lifecycle methods + decorators offer enough flexibility for third party libraries to express these kind of patterns and imho a third data source concept would only over complicate things.

    componentWillMount: function() {
        var baseRender = this.render;
        this.render = function() {
            if (this._watchDisposer)
                this._watchDisposer();
            var[rendering, disposer] = mobservableStatic.watch(() => baseRender.call(this), () => {
                    this.forceUpdate();
            });
            this._watchDisposer = disposer;
            return rendering;
        }
    },

@Mitranim I agree, that's a really good read, thanks for finding that! It's effectively what https://github.com/facebook/react/pull/3920 is suggesting.

We are undecided on which proposal is better. Having played with both, I'm mostly convinced that reactive programming (as you linked; https://github.com/meteor/meteor/wiki/Tracker-Manual) is the way to go, but we haven't reached consensus and we're still trying to figure out what makes the most sense, so feedback is welcomed.

@Mitranim @jimfb I've been a huge Meteor fan for several years now. Tracker is really cool and very fascinating. I created a presentation demonstrating how a very simple version of tracker works:

https://github.com/ccorcos/meteor-track/blob/master/client/main.js

And I even created an observable streams library with Tracker:

https://github.com/ccorcos/meteor-tracker-streams

But as I've fully transitioned to using React instead of Blaze, I've come to realize that Tracker is just too complicated and sometimes a simple publish/subsrcribe/onChange methods is just 100x easier.

Also, for all the functional programming fans, Tracker comes with some side-effects that make sense 99% of the time, but sometimes they get you. Here's how it would look in a React component

componentWillMount: ->
  @c = Tracker.autorun =>
    @setState({loading: true})
    Meteor.subscribe 'users', => @setState({loading: false})
    Tracker.autorun =>
      @setState({users: Users.find({}, {sort:{name:-1}}).fetch()})
componentWillUnmount: ->
  @c.stop()

My point about side-effects is that c.stop() will stop the subscription and the inner autorun as well.

@ccorcos Yes, in https://github.com/facebook/react/pull/3920 all the side-effects would be completely internal to React. In fact, React core could implement them in a completely immutable/functional way that would not perform any mutations. But that's an implementation detail, since the side effects wouldn't be externally visible anyway.

Interesting... So why not just use onChange callbacks? I've found those to be the most compatible. Whether you're using Meteor (Tracker), RxJS, Highland.js, etc., you can always integrate with them trivially with an event callback. And same with a React high-order component.

What I like about Redux is it keeps this logic out of the framework and keeps React components as pure functions.

@ccorcos The main problem is that when a component instance gets destroyed, all the "subscriptions" need to be cleaned up. Component authors often forget about this cleanup, thus creating a memory leak. We want it to be automatic, which makes it easier to write (less boilerplate) and less error-prone (cleanup is automatic).

@jimfb Exactly. Automatic unsubscription is one of the really neat things about Tracker's approach. Each call to the reactive data source re-establishes the subscription, and once the component stops calling for data, there's nothing to clean up!

Yeah, but its not really "automatically" unsubscribed. You have to call c.stop() in component will unmount. You can abstract that away with a mixin though.

componentWillMount: ->
  @autorun =>
    @setState({loading: true})
    Meteor.subscribe 'users', => @setState({loading: false})
    Tracker.autorun =>
      @setState({users: Users.find({}, {sort:{name:-1}}).fetch()})

But this is really no different from any other api. You can create a built-in method that automatically unsubscribes for you on unmount, etc. Look, I'm a huge Meteor fan. So I don't want to talk you out of this. Its just that sometimes, I find that its really hard to explain this stuff to someone who's unfamiliar with it. Meanwhile, using simple listeners / event emitters are much more simple to understand and implement, and they tend to be very compatible with whatever reactivity system you want to use...

@ccorcos We might be talking about different mechanisms. Last I checked, Tracker didn't have permanent subscriptions; each function that establishes a dependency on a reactive data source (by accessing it) is rerun _once_ when it changes, and that's the end of the subscription. If it accesses the data source again, this re-establishes the "subscription" for one more change. And so on.

@Mitranim is correct, the semantics of #3920 are more "automatic" than Meteor (unsubscription really is automatic), and far simpler since there is literally zero API surface area in the common use case.

@ccorcos @Mitranim For a ready to use Tracker / Vue.js inspired library you could try Mobservable, it observers all data accessed during _render_, and disposes all subscriptions on unmount (until then the subscriptions are kept alive). We applied it successfully to pretty large projects so far at Mendix. It is pretty unobtrusive to your data as well as it decorates existing objects instead of providing its own model objects.

Last I checked, Tracker didn't have permanent subscriptions

@Mitranim subscriptions can be permanent. Check this out.

sub = Meteor.subscribe('chatrooms')
# this subscription lasts until...
sub.stop()

Now there's some interesting stuff with Tracker. Check this out.

comp = Tracker.autorun ->
  Meteor.subscribe('chatrooms')
# this subscription lasts until...
comp.stop()

The last example isnt terribly useful though. Until we do something like this.

roomId = new ReactiveVar(1)
comp = Tracker.autorun ->
  Meteor.subscribe('messages', roomId.get())
# when I change the roomId, the autorun will re-run
roomId.set(2)
# the subscription to room 1 was stopped and now the subscription to room 2 has started
# the subscription is stopped when I call stop...
comp.stop()

each function that establishes a dependency on a reactive data source (by accessing it) is rerun once when it changes, and that's the end of the subscription.

The subscription lasts until the autorun is rerun (a reactive dependency changed) or the computation it lives in stops. Both of which call the computation.onInvalidate hook.

Here's a super contrived version that accomplished the exact same thing. Hopefully it will help you understand how Tracker works. And maybe you'll also see how messy it can get.

comp = Tracker.autorun ->
  Meteor.subscribe('chatrooms')

# is the same as

comp = Tracker.autorun (c) ->
  sub = null
  Tracker.nonreactive ->
    # dont let comp.stop() stop the subscription using Tracker.nonreactive
    sub = Meteor.subscribe('chatrooms')
  c.onInvalidate ->
    # stop the subscription when the computation is invalidated (re-run)
    sub.stop()
# invalidate and stop the computation
comp.stop()

@ccorcos I see the source of confusion now; it was my vocabulary. When talking about subscriptions, I didn't mean _Meteor subscriptions_. They determine what data is pushed from the server to the client, but aren't relevant to the view layer updates, which is the subject of this discussion. When saying _subscription_, I was drawing a parallel between traditional event listeners and Tracker's ability to rerun a function that has a dependency on a reactive data source. In the case of React components, that would be a component method that fetches data and then calls setState or forceUpdate.

@mweststrate Thanks for the example, it looks interesting.

Ah yes. So they have a clever way of doing it.

componentWillMount: function() {
  this.comp = Tracker.autorun(() => {
    let sub = Meteor.subscribe('messages')
    return {
      loading: !sub.ready(),
      messages: Messages.find().fetch()
    }
  })
componentWillUnmount: function() {
  this.comp.stop()
}

The subscription will just resubscribe each time with no issues. sub.ready and Messages.find.fetch are both "reactive" and will trigger the autorun to re-run whenever they change. Whats cool about Tracker is when you start to hide the autoruns and just have in the documentation that a certain function is within a "reactive context"

you could throw this in a mixin

componentWillMount: function() {
  this.comp = Tracker.autorun(() => {
    return this.getReactiveData()
  })
componentWillUnmount: function() {
  this.comp.stop()
}

And then you're left with this magically reactive function that just works!

getReactiveData: function() {
  let sub = Meteor.subscribe('messages')
  return {
    loading: !sub.ready(),
    messages: Messages.find().fetch()
  }
}

Tracker is pretty cool like this...

@ccorcos Turns out I was somewhat wrong about automatic unsubs when using Tracker-style eventing. You still need to stop the listener in componentWillUnmount, otherwise it'll keep reaching for data sources and calling setState (unless you check with isMounted() which is now deprecated).

On a side note, you can create elegant decorators to make component methods reactive. Here's some examples: [1], [2]. Looks like this:

export class Chat extends React.Component {
  @reactive
  updateState () {
    this.setState({
      auth: auth.read(),
      messages: messages.read()
    })
  }

  /* ... */
}

@Mitranim pretty neat -- I prefer high-order functions though. ;)

@sebmarkbage @jimfb I've been following this thread and the alt thread (#3858) for a few months now, and am curious if the core team has reached any consensus about this problem, or at least a general direction.

@oztune No updates; we've been focused on other priorities. We will post to one of the threads when there is an update on this topic.

Guys, I have a created a universal API to compose containers. Check react-komposer. Which that, we could create containers with a higher order function.

import { compose } from `react-komposer`;

// Create a component to display Time
const Time = ({time}) => (<div>{time}</div>);

// Create the composer function and tell how to fetch data
const composerFunction = (props, onData) => {
    const handler = setInterval(() => {
    const time = new Date().toString();
    onData(null, {time});
  }, 1000);

  const cleanup = () => clearInterval(handler);
  return cleanup;
};

// Compose the container
const Clock = compose(composerFunction)(Time);

// Render the container
ReactDOM.render(<Clock />, document.getElementById('react-root'));

Here's the live version: https://jsfiddle.net/arunoda/jxse2yw8/

We've also has some easy ways to compose containers with Promises, Rx.js Observables and With Meteor's Tracker.

Also check my article on this: Let’s Compose Some React Containers

@arunoda We wound up doing something very similar. One thing I'm wondering is how do you prevent composerFunction from being called on every prop change?

@oztune Actually now it'll run again. We use this Lokka and Meteor. Both of those have local caches and does not hit the server even when we call the composerFunction multiple times.

But I think we could do something like this:

const options =  {propsToWatch: ["postId"]};
const Clock = compose(composerFunction, options)(Time);

Any ideas?

@arunoda That's what we tried too, but it creates a bit of a disconnect between the action and its dependencies. We now do something similar to react-async, where, instead of immediately performing the given action, composerFunction would return a function and a key. If the key is different from the previous key returned by composerFunction, the new function will be performed. I'm not sure if this is a tangent from this github thread, so I'd be glad to continue on Twitter (same username).

@oztune I created a new GH issue and let's continue our chat there. Much better than twitter I guess.

Why not just let JSX to understand and render Observables directly so that I can pass Observable in props, e.g. this.props.todo$ and embed them in JSX. Then I wouldn't need any API, the rest is managed outside of React and HoC is used to fill in observables. It shouldn't matter if props contain plain data or an observable hence no special this.data is needed.

Additionally React render could be able to render Oservable[JSX] which would allow design described in the links without additional library.

https://medium.com/@milankinen/containers-are-dead-long-live-observable-combinators-2cb0c1f06c96#.yxns1dqin

https://github.com/milankinen/react-combinators

Hello,
As for now we could use stateless components with rxjs streams easily.
I dont understand the need for another API.
I wrote an example - a board that you can move mouse over it and when it gets to 26 it changes to restart.
I would love to hear what you think about this way.
Here is the code:
https://jsfiddle.net/a6ehwonv/74/

@giltig: I'm learning this way lately as well and like it. Together with Cycle.js.

I would appreciate to be able to listen to event handlers defined on components somehow easily without having to pass in Subjects for bridging. If I understand correctly it is pretty much the other way around than what is suggested here. Alternatively, if we could observe React vdom for its synthetic events, maybe "refs" could be used for observing to avoid having CSS tags just to observe.

Further RxJs could support object in combineLatest so that we could use React functional components directly to create larger components.

const myFancyReactComponent = ({surface, number, gameover}) => (
        <div> 
          {gameover ? gameover : surface}
          {number}
        </div>
)

const LiveApp = Rx.Observable.combineLatest(
    LiveSurface, DynamicNumberView, DynamicGameOver,
    myFancyReactComponent
)

Hi,
The reason we don't use Cycle or any other framework is that I prefer libraries over frameworks (more control to the developer) and also I want to enjoy the community power that React has.
So many render engines are now developed for React and it's a pity not to use it (ReactNative, ReactDom, ReactThree etc..). It's pretty simple to use just Rxjs and react as the example I've shown above.

It would have made thinks easier if react components could accept pojos as long as observables as props. As for now it's not possible so what I've shown above is the way we chose.

BTW what you did with MyFancyReactComponent is possible and we actually did that too in some cases though you can also write the jsx straightforward.

Regarding subjects - I think it's a valid way because eventually in React component I'm just using a handler function which could be anything. I choose to implement it with subject inside that receives events but someone else can choose otherwise - its flexible.

It would have made thinks easier if react components could accept pojos as long as observables as props. As for now it's not possible so what I've shown above is the way we chose.

observable props have a no sense in the long run. It has no sense at all in that context, actually..

It sounds to me like we ended up with a different model with Suspense (cache + context). The cache itself might get support for subscriptions. You can track remaining work for Suspense in https://github.com/facebook/react/issues/13206.

We also provide a subscription package for more isolated cases.