Where jet and other non-uniform maps are useful is in highlighting small fluctuations in the value. It's less about z1 > z2 and more about z1 != z2 across the image plane.
A similar problem happens with datashader in histogram normalisation mode. It's great for highlighting small variations in the underlying density and revealing detail that you'd otherwise never see, but worthless for trying to compare one density to another by eye.
I've had a few Rust lovers come and mention this project to me recently. None of them had any data science or ML experience. None of them knew that Python is just used to define the high level architecture.
At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a component that's idle 99% of the time.
The bulk of the load in an DL workflow is CUDA code and sits on the GPU. Even the intermediate libraries like cublas would only see marginal-to-none benefits of being reimplemented in rust.
This is a cool project, but it has no chance to displace or even complement Python in the data science space.
I am a data scientist and I care. The time when you could just do proof of concepts or a PowerPoint presentation is long behind us. So now we have to start to take it into production, which means we get the exact same problems as SE has always had.
Iff Rust helps us take it into production we will use it.
But it’s a lot of land to cover to reach Pythons libraries so I’m not holding my breath.
That said, Pythons performance is slow even when shuffling to Numpy.
I must be missing something. Modern data science workloads involve fanning out data and code across dozens to hundreds of nodes.
The bottlenecks, in order, are: inter-node comms, gpu/compute, on-disk shuffling, serialisation, pipeline starvation, and finally the runtime.
Why worry about optimising the very top of the perf pyramid which will make the least difference? Why worry if you spent 1ms pushing data to numpy when that data just spent 2500ms on the wire? And why are you even pushing from python runtime to numpy instead of using arrow?
Not everyone operates at that scale, and not every data science workload is DNN based
I agree with your general point, however, but the role I'd hope for with Rust is not optimizing the top level, but replacing the mountains of C++ with something safer and equally performant.
A big push for NN is to get them running in real time on local GPUs so we can make AI cars and other AI tech a reality. 2500ms could be life and death in many scenarios
Every small thing counts when you have big data which is exactly why you need performance everywhere, if Rust can help with that I don’t mind switching my team to that.
The problem are usually when you do novel feature engineering not the actual model training.
But I was a C++ dev before checking the assembly for performance optimization so I guess I have more wiggle room to see when things are not up to snuff. If I got a cent for every: -you are not better than the compiler writers you can’t improve this.
Especially from the Java folks. They simply don’t want to learn shit, which is fine if they just where not so quick with the lies/excuses when proven wrong.
This is just not true. The python runtime is not the bottleneck. DL frameworks are DSLs written on top of piles of highly optimized C++ code that is executed as independently from the python runtime as possible. Optimizing the python or swapping it out for some other language is not going to buy you anything except a ton of work. We can argue about using rust to implement the lower level ops instead of c++. That might be sensible though not from a perspective of performance.
In a "serving environment" where latency actually matters there are already a plethora of solutions for running models directly from a C++ binary, no python needed.
This is a solved problem and people trying to re-invent the wheel with "optimized" implementations are going to be disappointed when they realize their solution doesn't improve anything.
Yes. Let’s say you want certain features of a voice sample. You need to do that feature engineering every time before you send it to the model. Doesn’t it make sense to do it in C++ or Rust? This is currently already done. So if you already are starting to do parts of the feature engineering in Rust why not continue?
Yeah it’s not reasonable right now because Python has the best ecosystem. But that will not always be the case!
I can’t exactly tell what you mean but I think you’re confusing two levels of abstraction here. C++ (or rust) and python already work in harmony to make training efficient.
1. In tensorflow and similar frameworks the Python runtime is used to compose highly optimized operations to create a trainable graph.
2. C++ is used to implement those highly optimized ops. If you have some novel feature engineering and you need better throughput performance than a pure python op can give you, you’d implement the most general viable c++ (or rust) op and then use that wrapped op in python.
This is how large companies scale machine learning in general, though this applies to all ops not just feature engineering specific ones.
There is no way that Instagram is using a pure python image processing lib to prep images for their porn detection models. That would cost too much money and take way too much time. Instead they almost certainly wrap some c++ in some python and move on to more important things.
I know. That’s how we do it too. You don’t see any benefits in instead of Python wrappers + C++ just do Rust? Especially in handling large data like voice iff there was a good ecosystem and toolbox in place?
Maybe but then we’re no longer making an argument about performance, which is what I was responding to in your initial claim about “everything counts” and numpy shuffle being slow. That’s a straw man argument that has zero bearing on actual engineering decisions.
This smells like an overgeneralization. Often things that aren’t a bottleneck in the context of the problems you’ve faced might at least be an unacceptable cost in the context of the 16.6 ms budget someone else is working within.
In what circumstance would one measure end to end time budget in training? What would that metric tell you? You don't care about latency, you care about throughput, which can be scaled nearly completely independently of the "wrapper language" for lack of a better term, in this case that's python.
It seems some commenters on this thread have not really thought through the lifecycle of a learned model and the tradeoffs existing frameworks exploit to make things fast _and_ easy to use. In training we care about throughput. Thats great because we can use a high level DSL to construct some graph that trains in a highly concurrent execution mode or on dedicated hardware. Using the high level DSL is what allows us to abstract away these details and still get good training throughput. Tradeoffs still bleed out of the abstraction (think batch size, network size, architecture etc have effect on how efficient certain hardware will be) but that is inevitable when you're moving from CPU to GPU to ASIC.
When you are done training and you want to use the model in a low latency environment you use a c/c++ binary to serve it. Latency matters there, so exploit the fact that you're no longer defining a model (no need for a fancy DSL) and just serve it from a very simple but highly optimized API.
Looks good. I’ve tried Numba and that was extremely limited.
Current project we can’t use GPUs for production so we can only use it for development. Not my call, but operations. They have a Kubernetes cluster and a take it or leave it attitude.
We did end up using C++ for somethings and Python for most. I’d feel comfortable with C++ or Rust alone if there was a great ecosystem for DS though.
Latency in training literally does not matter. You care about throughput. In serving, where latency matters, most DL frameworks allow you to serve the model from a highly optimized C++ binary, no python needed.
quote is 'data spend 2500 ms on the wire'. That's not latency. For a nice 10GbE connection, that's optimistically 3 GB or so worth of data. Do you have 3GB of training data? Then it will spend 2500 ms on the wire to distribute to all of your nodes as part of startup.
the optimization you are describing is premature - you don't need rust to productionize your models and in most cases you don't even need to be coding in a low-level language at all.
I'm by no means a specialized data scientist, but I've done some (very surface-level) text crunching with both the TF/Numpy stack and Rust.
To me, the nice thing about switching to Rust for that kind of stuff was that it dramatically raised the bar of what I could do before reaching for those hyper-optimized descriptive libraries.
Want to calculate the levenshtein distances of the cross product of 100k strings? Sure, just load it into a Vec, find a levenshtein library on crates.io, and it'll probably be fast enough.
Could it be done in Python? Sure, but with Rust I didn't have to think about how to do it in a viable amount of time. Does that mean that Rust is going to take over the DS world? Probably not in the short term, Rust currently can't compete with Python's DS ecosystem.
But if I'm doing something that I don't know will fit into an existing Python mold (that I know about) then I'll strongly consider using it.
In fact, I'd basically argue against using most language-native implementations of algorithms where performance is at stake, because most implementations don't have all the algorithmic optimizations.
I think the industry is moving to 'MLIR' solution (Yes, there is a Google project called exactly that, but I am referring to the general idea here), where the network is defined and trained in one place, then the weights are exported, delegated to optimized runtime to be executed.
If such trend furthers down, then there will be very little reason to replace Python as the glue layer here. Instead it will become that everything is training in Python -> exported to shared format -> executed in optimized runtime, kind of flow.
Rust's opportunity could be to replace C++ in this case. But do mind that this is also a competitive business, where the computation is pushed further down to the hardware implementations, like TPUv1 and T4 chips etc.
Personally I hope for alternatives because I don't find Python that nice to work with compared to other languages. It's not awful, but I really miss having type and nullability errors detected by the compiler and reported to me with meaningful error messages.
Also there are a few weird things with the Python workflow, like dealing with Python2 vs Python3, and generally having to run everything in the context of a virtual environment, and the fact that it doesn't seem to really have a single "right way" to deal with dependency management. It's a bit like the story with imports in Javascript: those problems were never really solved, and the popular solutions are just sort of bolted on to the language.
It's amazing how many great tools there are available in Python, but it does sometimes seem like it's an under-powered tool which has been hacked to serve bigger problems than it was intended for.
> like dealing with Python2 vs Python3, and generally having to run everything in the context of a virtual environment, and the fact that it doesn't seem to really have a single "right way" to deal with dependency management.
Not to mention, this problem seems to be getting worse, not better. People are moving off of python 2.7, which was kind of the de-facto LTS release of python... leaving (currently) no LTS version of python and no clear path for the community to establish a new LTS version that the community will still support — there are so many releases with so many breaking changes in python 3 within the last few years that there is seemingly no consensus and no way to compromise.
> It's amazing how many great tools there are available in Python, but it does sometimes seem like it's an under-powered tool which has been hacked to serve bigger problems than it was intended for.
This is becoming more and more clear with every release of python IMO. The language is evolving too quickly for it to be used on large developments, but it’s still being used for that.
We have an entire test framework and libraries for high performance embedded systems level testing which is written entirely in python. The amount of high speed operations (timing tests, measurements, etc) is very obviously overwhelming for the intention of the language and libraries, yet the company keeps pushing ahead with this stuff. In order to mitigate the issue, we are developing more high speed embedded systems to offload the work from the test framework and report measurements to the framework. I think it’s quickly becoming extremely expensive and will only become more expensive — the framework is extremely “pythonic” to the point of being unreadable, using a number of hacks to break through the python interpreter. Jobs much better allocated to readable C++ libraries are being implemented in unreadable python spaghetti code with no clear architecture - just quick-and-dirty whatever-it-takes python.
I love python but I think it’s a quick-and-dirty language for a reason. What python does well cannot be beat by other languages (for example, prototyping), but I think it is often misused, since people can get something up and running quickly and cleanly in python, but it eventually has diminishing (and even negative) returns.
There are alternatives -- Julia and R both have bindings for Keras, which gets you Tensorflow, CNTK and Theano. (I think there are bindings for Tensorflow directly from R and Julia as well.) Once you have a trained model, it doesn't really matter from the production standpoint how you trained it.
Python should and will be replaced, but not at all for any of the reasons mentioned in this thread.
A good ML language is going to need smart and static typing. I am so tired of having to run a whole network just to figure out that there's a dimension mismatch because I forgot to take a transpose somewhere - there is essentially no reason that tensor shapes can't just be inferred and these errors caught pre-runtime.
Do you have an example of a tensor library that keep track of shapes and detect mismatches at compile time? I had the impression that even in static languages having tensors with the exact shape as a parameter would stress the compiler, forcing it to compile many versions of every function for every possible size combination, and the output of a function could very well have a non deterministic or multiple possible shapes (for example branching on runtime information). So they compromise and make only the dimensionality as a parameter, which would not catch your example either until the runtime bound checks.
If you explain a little more of what you mean, I might be able to respond more effectively.
> I had the impression that even in static languages having tensors with the exact shape as a parameter would stress the compiler, forcing it to compile many versions of every function for every possible size combination, and the output of a function could very well have a non deterministic or multiple possible shapes (for example branching on runtime information).
I was a bit lazy in my original comment - you're right. What I really think should be implemented (and is already starting to in Pytorch and a library named NamedTensor, albeit non-statically) is essentially having "typed axes."
For instance, if I had a sequence of locations in time, I could describe the tensor as:
Sure, the number of dimensions could vary and you're right that, if so, the approach implied by my first comment would have a combinatorial explosion. But if I'm contracting a TimeAxis with a BatchAxis accidentally, that can be pretty easily caught before I even have to run the code. But in normal pytorch, such a contraction would succeed - and it would succeed silently.
You understood it correctly. Named dimensions is certainly a good idea even in dynamic languages as a way of documentation and runtime errors that actually make sense (instead of stuff like "expected shape (24, 128, 32) but got (32, 128, 24)"). I hope it catches on.
But combined with static checking, it could be very very powerful. Definitely agree re: the power even in dynamic languages, I use namedtensor for almost all my personal development now (less so in work because of interop issues)
There's a research language that supports compile-time checking of array dimensions: futhark [1]. It's an interesting language that compiles to CUDA or OpenCL. However it's probably not ready for production (not sure if there's even good linear algebra implementations yet). It does feature interesting optimizations to account for the possible size ranges of the arrays (the development blog is very instructive in that respect).
Julia for example does have an array library that does compile-time shape inference (StaticArrays), but it cannot scale for large arrays (over 100 elements) exactly because it gets too hard for the compiler to keep track, I'm definitely curious about possible solutions.
But AOT/JIT compiled languages that can naturally talk to the GPGPU, without 2nd language syndrome, like Julia, Swift, Java and .NET will certainly be more attractive to data science practitioners.
I can already envision those life science guys that migrate to VB.NET when they have outgrown their Excel/VBA code, to start playing with ML.NET.
Python already gets JIT compiled to CUDA[1] and there's an entire funded ecosystem built around python+gpgpu called RAPIDS[2] which is the future of the ML space by most indicators.
I don't see any other language even making a dent in the Python ecosystem without some kind of new killer feature that can't be quickly replicated in Python.
I don't see anything in the article's python code that the numba's jit decorator cannot handle. When numba works (it's rapidly improving), it's seriously impressive.
For this particular case, you should be able to get really good performance without sacrificing readability.
> comparatively tedious languages like Rust will never attract data science practitioners.
Well, fast.ai is using swift now.
... I think it's fair to say 'never say never'.
You're probably right, rust isn't really the sweet spot for this stuff, but its also a case that python has some down sides that are pretty severe, and well acknowledged.
The parent comment literally said data science practitioners don't care about speed or safety because the GPU is where all the real work happens; that's false, I've provided an example of it being false from a respected party. What do you want me to say?
eh, I give up. Believe whatever you want to believe.
I am a practitioner, I care, but everyone on this thread seems to be distracted by the red herring of speed and efficiency.
None of these languages are going to be speedier in the GPU. But python has serious design problems that manifest itself when developing a large project.
> Because Swift for TensorFlow is the first serious effort I’ve seen to incorporate differentiable programming deep in to the heart of a widely used language that is designed from the ground up for performance.
> But Python is not designed to be fast, and it is not designed to be safe. Instead, it is designed to be easy, and flexible. To work around the performance problems of using “pure Python” code, we instead have to use libraries written in other languages (generally C and C++), like numpy, PyTorch, and TensorFlow, which provide Python wrappers. To work around the problem of a lack of type safety, recent versions of Python have added type annotations that optionally allow the programmer to specify the types used in a program. However, Python’s type system is not capable of expressing many types and type relationships, does not do any automated typing, and can not reliably check all types at compile time. Therefore, using types in Python requires a lot of extra code, but falls far short of the level of type safety that other languages can provide.
...But fast.ai is a Python library. Partially written in swift.
Validating the original point that nothing will replace python for DL applications any time soon but middleware will continue to be implemented in c++/rust/swift/whatever you fancy.
S4TF isn't the first and certainly not the last end to end non-python DL stack. It might be worth highlighting as an example if it ever reaches mindshare above the noise floor amongst those stacks.
> Our hope is that we’ll be able to use Swift to write every layer of the deep learning stack, from the highest level network abstractions all the way down to the lowest level RNN cell implementation. There would be many benefits to doing this...
Well, the tldr: you’re wrong.
The more approachable reading: python isn’t going anywhere, but people are looking at other things for more than just low level implementations with a python wrapper.
...it’s early days yet, who knows where things will go... but maybe do a bit more reading and have an open mind?
Python is easy to get started with, but once a project grows to any meaningful size, I would rather have a compiler which is giving me more correctness guarantees than Python is capable of.
IMO Swift strikes the best balance between strictness and productivity of any language I've worked with.
> Python is easy to get started with, but once a project grows to any meaningful size, I would rather have a compiler
boy do I have a continent full of python developers to introduce you to... “python - by any means necessary” is their motto.
But in all seriousness, there are a lot of people (where I work) who started with python and have been daily driving it for so long that any other language is too tedious to get started with. Even if it means writing unreadable, hacky python, python still wins out for them.
I suspect there are a lot of similar people in data science.
Yeah there is some tedium in general with Rust syntax. Semicolons, for example, feel old fashioned, and there's a lot of verbosity in things like unwrapping. It's a fine language and I like working with it, but there's a lot of details involved in Rust development which don't make sense for data scientists to worry about.
This is why I think Swift is a much better choice to replace or at least compliment Python than Rust. It has a modern, powerful type system and all the quality of life advantages which come with it, but it manages this with a lot more usability than Rust.
A well written swift framework almost becomes a DSL for the problem domain, which is a great property for a data science tool to have.
Swift is not fun on non-Mac platforms. It feels a lot like Google's Dart in terms of how it was positioned and advocated.
Modern Rust isn't difficult to use. This is becoming a really tired meme from detractors. The compiler is incredibly helpful, non-lexical lifetimes are a thing, and unless you're doing a lot of sharing and parallelism, you can avoid many borrow checker problems until you learn RAII.
It's a fair criticism of Swift that it is not well supported across platforms. The story is getting better, but it is still miles behind Rust in this regard. I am a huge fan of Swift as a language, but I am very critical of the tooling.
> Modern Rust isn't difficult to use. This is becoming a really tired meme from detractors.
I beg to differ. I've been programming professionally for over a decade, and I have shipped projects in a variety of languages, and I can safely say that Rust has a steeper learning curve and requires more cognitive overhead to use than many other languages. I find it relatively nice to work with rust in spite of this because the tooling is so great, but it's undeniable that Rust has made tradeoffs which sacrifice ease of use in favor of safety and performance.
The tooling for Swift 4 TF is not anywhere near satisfactory. I can't even seem to get it on my computer without installing XCode (and that's on an apple computer)
I should be able to pull a docker image and have s4tf immediately at my fingertips
The Dockerfile in the swift-jupyter repo is a superset of what you need. You could remove the lines dealing with jupyter and you'd be left with a Docker container with the s4tf compiler.
The "Rust is difficult" criticism isn't getting tired, but it must be getting tiresome for the Rust community because they don't have a good answer to it.
The top level comment from an early adopter in the last Rust release thread on HN was complaining about the tedious complexity.
I think this project has immediate merits when it comes to productionizing small, sparse networks for running on the CPU.
I do research and prototyping in Python, but I have to deploy on mobile devices. I was going to roll my own implementation, but now that this exists, it's something I'm going to look into.
That made it really clear that texture resolution trumps mesh resolution. Dialling down the mesh detail slider results in very little perceptual difference, but the blurry texture always stands out.
The non-textured demo actually looks better. Nothing that can't be fixed by stitching higher resolution sat imagery though!
Would love to learn more, since it seems counter-intuitive — are there any studies / benchmarks on this for modern hardware? Why would a uniform mesh with, say, 10x more triangles perform better if it's not updated often (e.g. a few times a second)?
Mainly because GPU throughput is increasing much faster than CPU throughput. Spending cycles removing triangles doesn't matter if the GPU can just churn through them like nothing. In the context of a moving camera you want to reuse as much data as possible between terrain updates. With regular nested grids a ton of data can be reused, only the height needs updating as you move the grids around (and height can be reused between adjacent grids half the time if they have a power of two resolution relationship).
But baking irregular meshes into a tree-structure could be powerful too I suppose, I'm not that familiar with algorithms involving them. All of this is highly dependant on actual size of terrain and LOD requirements, terrain rendering is a deep rabbit hole.
Yeah, it's a thin balance, and to me as someone without years of experience in graphics programming, GPU feels like such a blackbox — hard to predict what will be the actual bottleneck until you implement an approach fully, with real-world loads, and even then it's hard to assess, especially with such variety of GPU performance characteristics on different devices.
So what's simpler / easier to implement in a given system will play an important role, and so are any additional things you want to do with the height data like collision detection, querying, data analysis etc. Anyway, that's a super-exciting topic to learn more about! Thanks a lot for all the thoughtful comments.
It'll come down to how frequently you're doing updates. A few times a second will certainly cost more than a prefab mesh with a texture fetch shader.
If you can precompute various LOD chunks for each tile and cache them, under some circumstances that's optimal, but I don't think any of them are webgl. A typical heightmap based terrain renderer today will consist of several (3-5) prefab LOD meshes deformed by height map textures in shader, like[1]. Swapping a textures in and out of vram is cheaper than swapping geometries, even if those geometries are of lower resolution.
Modern GPUs are almost never bound by the number of triangles on screen. The bottleneck will be the fillrate and bandwidth and pipeline stalls, especially in webgl. If you can throw down more triangles to do less of the other things, it's almost always worth it.
I have a toy mapbox tile renderer based on this approach. I'd love to know if you guys have plans to implement full 3d like google maps any time soon.
I would really love to see your toy Mapbox renderer — please share it, even if unpolished! We'd benefit a lot from fresh ideas on how to improve Mapbox GL performance.
GPUs are effectively “magic” and can do particular things hundreds or thousands of times faster than CPUs. Offloading a per-vertex or per-pixel task to the GPU, especially if doing so reduces bandwidth requirements, is almost always a big win.
I think reality is closer to this: GPU rasterization is 20-50 times faster than CPUs and throughput computing maybe 5-20 times faster.
For example consumer Ryzen 3 with 12 cores peaks at 32 FLOPS/core/cycle. So at 3.8 GHz, peak ~1.4 SP TFLOPs (or 0.7 DP TFLOPs). But just 50-60 GB/s memory bandwidth limits it somewhat.
Quick googling says consumer Nvidia RTX 2080 peaks at 10 SP TFLOPs (or 0.314 DP TFLOPs, yes, less than half than the CPU example). Memory bandwidth being at 448 GB/s.
GPUs win massively at rasterization, because they have huge memory bandwidth, a large array of texture samplers with hardware cache locality optimizations (like HW swizzling), texture compression, specialized hardware for z-buffer tests and compression, a ton of latency hiding hardware threads, etc.
But they're definitely not thousands or even hundreds of times faster.
Not very helpful if you need to do collision on the displaced mesh though, I would have thought, since a vertex shader only make it "appear" as if the vertices are not in their original position?
I'm thinking of tracing for foot placement in conjunction with inverse kinematics for walking characters, or vehicle wheels traversing the terrain. I've run into this problem with UE4.
Would it be possible to either compute or approximate the fragment shader's vertex results on segments of the terrain that require collision detection? I bet it wouldn't be feasible to do to check collisions for gun projectiles in a video game, but it might be feasible for something like making sure the player's model doesn't clip beneath the tessellated terrain.
UE4 has adaptive tessellation anyway, and you can scale the tessellation multiplier based on distance with a material function so that the landscape has more triangles closer to the camera and less further away. Beyond a certain distance (where the player wouldn't notice the detail anyway, so there's no point drawing it) you can scale the multiplier to zero to improve performance.
I'm not an expert but honestly in a game engine like UE4 where collision is important, it seems like the adaptive geometry tessellation approach is the way to go, as least from my initial experiments.
I was thinking of a somewhat different use case. I was thinking of a flight simulator where tessellation only occurred close to the camera, to provide the appearance of a more detailed mesh when viewed up close.
The problem is, with a quad mesh where each vertex is 20 or 30 meters apart a plane flying nap of the earth could end up clipping through the tessellated mesh but not actually collide with the non-tessellated terrain quad mesh being used for collision detection. Or worse, the player could collide with the terrain quad mesh but the tessellation makes it visually appear to have not collided with the terrain.
To only two ways I see to solve this are to either not use tessellation up close, or to compute with the CPU the vertices produced by the tessellation and use that mesh for collision.
Geometry shaders seems sort of the bastard child of the graphics pipeline. It was a mess to implement in hardware and the throughput isn't that great when creating tons of triangles. The tesselation stages of the pipeline addresses this and is more suitable for arbitrary subdivision of triangles.
Yeah. If the thing is not at least in an X8 configuration (2 independent motors and props for each edge) with enough thrust to safely land you in X4, it should never have passengers on board.
The car shown in OP is one motor/esc/prop failure away from killing whoever is inside it.
Cool story. But when I get a replacement battery pulled from a brand new iphone with the 3rd party repairer providing warranty on their work, I still have no way to make that message go away other than to pay Apple's extortion fees.
Fun local fact: unless you order your battery at iFixit and DIY, it’s cheaper to go to the Apple Store to change an iPhone battery than any of the repair shops nearby.
Fun reality check: for many people, going to an Apple store means driving several hours, or even purchasing a plane ticket. Any reasonable accounting of that would massively inflate the cost of 1st party repair.
.. not if you go to an authorized Apple repair center. Unless you live in some wilderness area, there is almost certainly an authorized repair center near you — and those repairs are backed by Apple. Going to some sketchy shop that cannibalizes new phones to sell a replacement battery? That is bizarre. I am starting to understand why Apple has apparent hostility towards unauthorized third party repair. Someone is likely going to go buy one of those “new” phones and think Apple is crap. Meanwhile, the repair shop made money selling you the battery from a new iPhone, then money selling a fake-new iPhone with a garbage battery to someone else.
You're seriously living in an urban bubble if you think either an Apple store or a certified repair center will be readily available and convienent for everybody. Let me remind you, the alternative is Apple sends me a new battery in the mail and I insert it into my phone myself. Zero downtime or travel.
The last time I got a battery replacement for an Apple product was a few months ago, when I was living in Seattle a mere 30 minutes away from the University Village Apple store. From the time of my first phonecall to schedule the appointment to me walking out of the store with a new battery, three and a half weeks elapsed. I had to go to the store three times, the first time they merely "confirmed" what I had already told them on the phone and said they'd need to order the battery and it would take a week. What a fucking disaster, why didn't they order it when I called? Why did they give me an appointment when they knew they couldn't help me that day? Imagine if I was living an hour or two out of town, that would have been three afternoons of my short live wasted. Unacceptable.
It's shit. Unmitigated shit. With my old thinkpad I had new batteries mailed to me and it took mere seconds for me to swap them. I am never buying another computer from these shitheads. People who continue to support Apple have either gotten lucky, or they're nuts.
Why would a repair shop rip a battery from a brand new phone to sell the battery to you? They ruin a new phone to sell you a $20 replacement battery? Now what will they put into the formerly new phone without a battery? Third party junk that some unsuspecting buyer is going to buy thinking it’s a “new” phone?
No. That's exactly what Apple are doing. Out of all the ways they could harm the consumer in this situation, that message is by far the most effective. Stop defending them. Stop giving them benefit of doubt. Stop misconstruing their malice for incompetence. Apple has repeatedly demonstrated that they'll go to extreme lengths to harm consumers and prevent 3rd party repair.
That message destroys the consumer's trust in 3rd party repair shops. It says battery health issue. Are sure you put in a real Apple battery? Did you even change my battery at all?
Outright locking the battery out would cause backlash and possible legal action. This 'technically works but will nag you forever unless you pay Apple to run a program to clear the message that they refuse to share with 3rd parties' solution is genius. Evil genius.
Imagine the car equivalent of this situation. Imagine you take your BMW to a non-BMW-authorised repair shop and they swap your battery to a perfectly good one for 1/5th the price. But now you have a permanent warning light on your dashboard that there's something wrong with your battery. This situation is actually impossible. There are laws that require carmakers to release repair manuals to 3rd party repairers and honour warranty after 3rd party repairs. Tech companies are shitting on their users because equivalent laws don't exist for electronic goods.
> Imagine you take your BMW to a non-BMW-authorised repair shop and they swap your battery to a perfectly good one for 1/5th the price. But now you have a permanent warning light on your dashboard that there's something wrong with your battery. This situation is actually impossible.
You wouldn't believe it, but that's almost exactly how it works with bmws since around 15 years. The unofficially replaced battery won't function properly until "registered /converted/(or even) programmed" [0] at the official bmw service. The difference is that the software to do that is pirated and thus available to the 3rd parties
3rd party repairers have the ability to do this for you because they have BMW-compatible programmers. Because BMW must release make that possible for them. By law.
It's also there for a reason: the charging system needs to be told that the battery was changed and what type of battery it was changed to. Otherwise it charges it the wrong way. Unlike Apple batteries, the car batteries are dumb and can't tell the car about themselves.
The warning light won't come on just because you replaced a battery. It'll come on a few days later if you replaced a battery and didn't tell the system about it so now the battery is performing poorly because it was being charged incorrectly.
3rd party manufacturers, by law, have access to the programming tools required to do this for you. Apple holds the equivalent of these programming tools away from everyone and aggressively sues anyone that manages to obtain them or reverse engineer them.
I guess a better analogy would have been '3rd party repair shops installs genuine BMW battery but is unable to register it because the tools to do so are held hostage'.
There are 3rd party tools to register the battey not only pirated software, most automotive software now uses the same interface (j-2534 passthrough) so you pay to download the needed files for programming which is totally acceptable. Apple instead have a battery ransomware.
A better metaphor would be a service warning when you go into the iDrive system, to the "CAR" submenu, scroll to the maintenance icon, go into the submenu and then ask the system to list all possible issues. Then it would show up.
Also a 1/5th of the price is not entirely fair. The difference between Apple and 3rd party shops is half at most. And even that is not a fair comparison, because the non-BMW-authorised repair station will use an OEM-equivalent battery from VARTA or BOSCH. Your iPhone repair shop will use a random battery imported from AliExpress with no know history of it being a safe battery.
I am on the fence. On one side I would love if it were possible to repair your phone whenever wherever.
On the other hand, let me tell you a personal anecdote. After my wife’s iPhone got stolen we wanted to buy another one for cheaper. We chose to buy it from a big retailer in here. The phone was marked as renewed and under warranty. It cost a bit less as a refurb from Apple would.
When we got it it was immediately apparent that the screen was changed for a non first party one. The phone was thicker than original (a case would not fit) and the colors were shit.
No warnings were displayed on the phone. I have returned it immediately for refund.
Now, if somebody who does not know how an iPhone should look and behave it is quite possible that they would pay a lot of money for a subpar product and then tell about it to people around.
I think Apple should absolutely put in warning lights for any non genuine components or genuine components installed by non authorized repair shops.
But, the message here is off as it does not actually help to describe the problem.
The phone knows that the battery is a genuine Apple component, because the battery cryptographically identifies as one. What it's detecting is that a genuine Apple battery was installed in the phone, but not by Apple.
It displays the warning to scare the user and make them mistrust the 3rd party repairer. The only way to make that warning go away is to use a secret-sauce Apple programming tool that they withhold from 3rd party repairers and sue anyone that manages to reverse engineer it.
They are not doing this to protect you. They're doing this to make you distrust 3rd party repairers, or to make you avoid the nag warning, and go to their overpriced store instead.
I'd probably take objective measurements of physical properties over a "profile new battery" charging cycle, and not solely rely on supply-chain QA personnel, protected by DRM nonsense.
My charger for rechargeable AA (LR6) batteries can do that, recondition batteries to eke out a few more cycles, and also tell me when a battery is finally gone beyond its powers to revive.
And my charger is not a $1000 device that utterly relies on the health of its battery to function, either.
What Apple is doing is slapping an opaque cover over the report screen until someone comes along that has a company-issued "remove report cover" pass. What if someone put alkaline batteries in there, instead of metal hydride? What if they put in AAA batteries instead of AA?
The charger can detect wrong chemistry types, like alkaline and NiCd, by objectively measuring the physical properties. And it can still charge AAAs. They just have a lower capacity. The battery subsystem in the phone could analyze a new battery and report its condition, but that would require Apple to admit to itself that a battery is a 3rd-party replaceable part that will require replacement some time over the projected lifespan of the device.
How do you know? You measure. High-tech batteries have some of that capability built-in, and high-tech devices with charging circuits connected to high-powered general computing processors can certainly run automated tests after their case-intrusion sensors and/or power-interrupt sensors detect events.
Then it has a smaller capacity than a brand new battery should, but still larger than a heavily used battery. It's something the phone can report on, and android phones do. Apple can't guarantee your replacement battery was 'stored properly' either. All they can do is run their little programming tool to remove the warning regardless.
This scenario is completely outside the realm of 'new genuine Apple battery detected + no Apple warning reset = permanently harass the user with false warning'.
Let's be really clear here. The warning is 'unable to verify this iPhone has a genuine Apple battery'. Which is complete bullshit. The phone is perfectly aware that the battery is genuine because it carries a crypto signed identifier.
So if Apple rephrased this message to state, “unable to verify the health of this battery,” but still had the exact same system response, you would be happy?
I think the message could have been clearer. But to add to your point, wasn't there a case when car repair shops would rewind odometers on cars/engines to make them seem newer that they actually were?
> Now, if somebody who does not know how an iPhone should look and behave
If you don't know exactly how an iPhone should look and are not annoyed by the thickness difference, how is it a problem ? You paid less for a slightly less good product that's good enough for you, and if the product is not good enough for you (like in your case) you returned it and good a refund. That sounds pretty good for me!
No. I paid slightly less for a very inferior product (non compatible with cases due to thickness, colors off by a large margin).
In this case I knew that I got a bad product and returned it, but it makes me wonder how many times I got some shoddy fake instead of the genuine thing (e.g.: because of Amazon stocking shenanigans). My reaction to bad products is usually to distrust the seller and the manufacturer brand.
This is also why brands fight fakes, if you can get a visually extremely similar object that is sold as product made by B, but it falls apart after a few months then there is a big chance that you won't buy B anymore even though their products are actually good.
Also in my case the problems were visible. If the problem was with a badly repaired internal component, the phone would have died, I would have brought it to Apple and they would refuse to repair it under warranty because it was voided by some random technician. Had the repair been done badly by an authorized repair shop, they would take the cost of re-repair or exchange on themselves.
Apple burying a message in the battery subsection of settings is "extreme lengths" and "by far the most effective"? God help us if they discover how to log to console.app.
How is it burying when it’s probably the first thing someone would look at after a battery replacement.
Also, I bet none of the Apple defenders were phrasing the location of the battery health meter as “burying” it when Apple released it as a response to their own battery fiasco (which they liead about for months, if not years, until they were proven to have lied).
I am a cheap customer using the fifth battery in my very old Nexus 5.
What Apple says is exactly correct. Battery replacements never hold charge as well as the original battery (in my case no longer manufactured). And spare batteries are of lesser quality.
They are also very cheap, about $9, so I don't care if I have to charge the phone twice every day, until it dies.
However, knowing the sue-for-anything culture in the USA, I back Apple on this one. The message about the battery is accurate, according to my own experience.
Now, the shenanigans with their special screws and tools, the everything glued inside, and removal of the audio plug, I will always be against.
Yeah, uh, BMW sucks and so a lot of auto makers. Instead of making an additional menu in the ridiculously over-complicated dashboard computer system, I had to buy a $150 tool that can do a "battery registration" so that the smart alternator knows that there's a new battery installed and not to charge it so much. You don't have to buy BMW brand batteries, though. But Jesus BMW, get over yourselves. There could be a button right by the battery in the trunk to do this, it doesn't have to all be locked down.
That figure doesn't sound convincing, and the linked paper seems to just make it up without any explanation or justification.
The limiting factor of a quad's ability to fly is the motor and unexpected failure of the battery. Higher end brushless motors have a life expectancy in the tens of thousands of hours. Battery failure is already mitigated in consumer drones by having independent packs each of which provides enough current to continue operation.
Quads do occasionally just fall out of the sky because a defective bearing ceases or a defective cell sags, but it's very rare. Most of the time they crash due to operator error.
You've left out the critical component between the battery and the bldc motors: the motor controller. These deal with the most difficult thermal issues and fail frequently.
On a quad there a 4 motor controllers on board, and only one needs to fail, greatly increasing the odds of overall failure.
I've never heard of anyone having an ESC fail mid flight. Again, it might happen, but the mean lifespan of a healthy ESC is definitely >100hrs.
None of this is even really relevant. GP quoted a figure from a paper that pulls the figure out of thin air and doesn't justify it in any way. The figure can be disregarded. I'm just pointing out that the figure seems much lower than mtbf of any of the critical components of a quad.
Considering that mechanical failure is basically a rounding error compared to human error when it comes to causing crashes of every other vehicle and piece of machinery humans operate I'm inclined to believe the same is true for drones.
People have a much lower tolerance for mechanical failure in vehicles that carry people. Anything that crashed at a higher rate from mechanical failure than human error would not be produced for very long.
> Exports are less important to GDP than real estate construction and infrastructure development (subways, high speed rail, bridges, etc).
Exports have stopped mattering several years ago. Manufacturing is increasingly being moved to Viet Nam, India, and Africa. Countries with cheaper labour costs and that don't steal your IP and lock you out of their domestic market as part of the manufacturing deal.
You forgot the ghost cities, the buildings that crumble before they are ever populated, and the roads that fall apart within a year of being built. The backbone of the Chinese realestate economy.
The chinese real-estate investment bubble is a bomb and the trade war is rapidly counting the timer down to 0. Is it a bubble? Yes it is, the Chinese government has had to put in multiple controls over the last 5 years to slow its growth and try to prevent it from bursting[1]. They're in a difficult position. Letting it keep growing will guarantee it bursts. Stopping it entirely will cease only source of the economic growth that's kept the population placated with the authoritative government.
The more wealthy and smarter Chinese have been investing overseas. That investment has fallen exponentially since the start of the trade war[2]. This bubble isn't going to hold up for more than 2 more years, assuming the trade war continues. And if it bursts, it'll hit the world economy pretty hard.
Recently the Chinese government has also put in tight controls on who can move money out of the country and how much. This was in response to everyone trying to move all their assets overseas as the trade war ramped up. They're even going after party members now[3].
tl;dr: China's economy is large, but most of it is a house of cards built on a bluff. That bluff has been called during a downturn that the CCP didn't know how to address in the first place. Anyone with the means to do so is folding, cashing out, and leaving the country or at least sending their children overseas[4].
>The more wealthy and smarter Chinese have been investing overseas. That investment has fallen exponentially since the start of the trade war[2].
I’ve been around the world enough to see this first hand. Empty brand new offices in Malaysia, Taiwan, Indonesia. Big apartment investments in small cities in USA and Europe. Lots of both in the southern half of Africa.
Perhaps the most interesting were the giant empty office buildings in Asia. Brand new, but seemingly falling apart, elevator shafts open with wires hangout of control panel locations. Ready to fix up for the right client, but I wonder who that is and when they are coming?
Crucially, if the so-called house of cards in China does crumble, we're likely to see residential and commercial real-estate suffer worldwide. This is the consequence of years of capital flight coming out of the country.
The luxury market in Seattle (Bellevue) crumbled after the Chinese foreign asset taxes went into place. Empty homes that were Chinese owned, are still largely Chinese owned, but suffering from deflation of market value. Home construction plans have slowed or stopped and luxury vehicle, jewelry, etc stores have folded up.
Where jet and other non-uniform maps are useful is in highlighting small fluctuations in the value. It's less about z1 > z2 and more about z1 != z2 across the image plane.
A similar problem happens with datashader in histogram normalisation mode. It's great for highlighting small variations in the underlying density and revealing detail that you'd otherwise never see, but worthless for trying to compare one density to another by eye.
Horses for courses.